The reason that markets do not appear to work in healthcare is that there really few if any true markets deployed in healthcare. The question is, what are true markets and why do they tend to allocate resources so efficiently.
The realization I recently had focuses on a concept first put forth by Amos Tversky and Daniel Kahneman called loss aversion. Basically loss aversion means that people prefer avoiding losses as opposed to making gains. Any true market transaction must deal with loss aversion because market exchanges mean giving something up first in order to gain something. There is a complementary element to this called the endowment effect which means people place a higher value on what they already own.
What does this have to do with markets and specifically health care markets? For a transaction to happen in a free market, those undertaking the exchanges must view that they will be better off after the exchange occurs. That is a very high bar given these realities of loss aversion and endowment effect. Both parties need to hold a similar perception that they will both be better off after the exchange. Each will have to give up something they own and in market systems, what they give up is very clear to the parties undergoing the exchange. The initiation energy for the transaction has to overcome the loss aversion barrier, meaning that both parties need to be confident they are much better off after the exchange.
Health care markets are flawed markets at best. The major financial transactions happen on the front end before any services are even contemplated. They are in the form of deductions from salary, taxes, or premiums paid. These transactions do not pay for any specific health care service and they pay for some set of services which are poorly defined. Many if not people never see the money in the first place and really have no idea of what they have purchased.
This was constructed this way intentionally because if people were required to pay for premiums on a monthly basis they would experience loss aversion and push back on payment. Even with nominal charges for copays and deductibles, loss aversion prompts people to stint on their own care for the simple reason that they may perceive little value in what this money goes toward. The copays and deductibles are small fractions of the overall dollars washing around the system.
This is not unique to healthcare. Any long term investment generally has less than perfect buy in from the general public. People do not save for their retirements and Social Security is a form of forced savings. OK, it is not really savings and it is actuarially unsound, but the reason it was put in place is that because of the propensity of humans to not save for the future.
The use of insurance to pay for health care is another form of forced savings to pay for what most people will not or can not plan for. We could try to deploy more market based approaches to pay for health care but they are likely to fail miserably, or at least be viewed as failures, but not for the reasons you might think. They will fail because if people given adequate resources and are placed in charge of how they are spent, they will not view that investment in health care brings them sufficient value to invest the resources they control. We do not trust markets in health care because we do not trust people to make the decisions that we view are best.
Let's say that instead of providing subsidies for health insurance we simply provided subsidies that people could spend as they please. We are probably right that large numbers would not spend their new found gains on health care if they were given a choice. Are those who fail to make decisions that we feel best for them short sighted of are we simply arrogant to assume that we know best?