Sunday, November 8, 2015

Markets fail. So what?

In welfare economics, a market failure is when the competitive price system fails to allocate resources efficiently, where this usually refers to a violation of Pareto optimality. This means there are unexploited ways to make some people better off without making anyone worse off. If, for example, the market systematically underprices a good because some of the costs associated with its provision are externalized on the public, that’s a market failure. If the market under-provides a good because there isn’t a good way to prevent free riders, that’s a market failure.

A very common strategy of argument is to identify a market failure and then suggest a government intervention designed to address it. For example, the standard microeconomic analysis of public goods provision suggests that things like lighthouses would be under-provided by the market because of people’s propensity to free ride. Someone might notice that ships need lighthouses and build with the hope of signing up paying users, but this arrangement would certainly fail because lighthouses are non-excludable and non-rivalrous. None of the ship owners would pay for the service. If we’re going to have lighthouses, we need the government to provide them.

But market processes aren’t the only kinds that fail to secure an efficient outcome. Theories of government failure developed among public choice economists in response to the assumption that in a case where a market process has failed, a government decision-making process will correct it. This is closely aligned with the assumption that government actors are genuinely benevolent and reliably motivated to pursue the common good. Public choice theory shows that you can generate better predictions of government behavior by assuming that people holding government offices are people of normal good will and largely motivated by self-interest.

Just like there are several well-theorized sources and examples of market failure (e.g., externalities and public goods), there are likewise several well-theorized sources and examples of government failure. In cases of corruption, government officials use their control of public resources to advance their private ends. An official may be in charge of some project and solicit bribes in exchange for granting the government contract supporting it. The problem here isn’t that it’s immoral, though it is. The problem is that extending a contract on the basis of someone’s willingness to provide a bribe will almost certainly violate Pareto optimality. Public choice theorists argue that ineffective monitoring regularly permits politicians to benefit themselves at the expense of the public. Individual losses among the public may be quite small. In fact, that they are small explains the ineffective monitoring since their losses escape their notice. Therefore, their ignorance about who it is best for them to vote for, what policies are best for them to support, or who might be taking advantage of them is rational. But in the aggregate, their total losses will tend to be much greater than the benefit the politician consumes in the form of rents.

This dynamic of concentrated benefits and dispersed costs also figures into accounts of regulatory capture. When political actors have a great deal of discretionary power, this generates powerful incentives for an industry to use whatever means available to influence the decision-making process. They might convince regulatory agencies to permit certain profit-enhancing externalities or provide economic protection from foreign or domestic competitors. These high stakes provide incentives to win influence that are much stronger than anything that would induce an individual citizen to organize with others to help keep the regulatory agency’s activities in line with the public interest.

So governments fail, too. We should, therefore, be on guard against committing the Nirvana fallacy. The following syllogism makes the mistake in the Nirvana fallacy pretty obvious:
 1. In a range of circumstances, markets constrained by interventionist policies administered by morally and informationally perfect people would have better outcomes than markets free of any interventions. 
2. In those circumstances, actually implementing those interventionist policies administered by morally and informationally perfect people would have better outcomes than the market free of any interventions. 
3. Therefore, we should implement the interventionist policies.
Obviously, 3 does not follow from 1 and 2.

This lesson, and even many sources of government failure, was acknowledged by, of all people, Cambridge welfare economist A.C. Pigou, who is the patron saint of market failure theorists. As early as 1912 in Wealth and Welfare, he wrote
“It is not sufficient to contrast the imperfect adjustments of unfettered private enterprise with the best adjustments that economists in their studies can imagine. For we cannot expect that any State authority will attain, or even whole-heartedly seek, that ideal. Such authorities are liable alike to ignorance, to sectional pressure, and to personal corruption by private interest.”
Again, markets fail. But even when they do – even when real-world markets do not meet the standard modeling assumptions that ensure perfect competition and Pareto optimality – government intervention may make things worse. The government is, at best, another tool societies can sometimes use to good effect. It is not a Deus ex machina that societies can rely upon to swoop in and bring about a happy ending.

The possibility of government failure should militate against the tendency to compare the reality of unregulated markets with an idealized implementation of government control in order to argue for interventionist public policy. That isn’t the choice that’s available to us. Instead, we have to choose between the messy real-world outcomes of unregulated markets and the messy real-world outcomes of regulated markets.

Examples of messy real-world institutional arrangements might actually surprise some economists in the way outcomes sometimes do not cooperate with standard microeconomic models. Return to the lighthouses. In 1820 about 75% of lighthouses on the English coast were built and operated by private parties because they could effectively limit access to their service by tying its use to entry into harbors. There, berths were excludable and fees were easy to collect. This example may suggest a sort of market resiliency where cooperative solutions to market failures emerge without government intervention because novel solutions are incentivized by mutual gains from trade.

Government failures generally don’t have this natural self-correcting feature, which may make them more serious. To correct a government failure there must be someone with the insight to devise a solution and the benevolence, courage and skill to see it through in the face of highly motivated political opposition. But politics eats up people like this for breakfast.

Kyle Swan
Department of Philosophy
Sacramento State


  1. Sir, there is an obesity epidemic in this country, and you are warning about the dangers of insufficient body fat. It's not that what you say is false; it is just that overconfidence in the benefits of government intervention is such a rare ailment in this country and age, warning against it is not going to do much for the vibrancy of the intellectual debate.

  2. Kyle, this is interesting and persuasive, thanks. But I am wondering if it is slightly more persuasive than it ought to be. Markets seem inherently inclined to failure. But what you say about government failures seems to be based on contingencies that vary a great deal from place to place and time to time. Of course, the incentives are universal, but corruption in government officials, for example is something that is a far worse problem in some countries than others, and seems to have been a worse problem in our country than it is today. So evidently we have the ability to correct past mistakes to some degree and make government failure a little less likely than in the past.

    Of course, part of this correction just involves doing less of it, recognizing and avoiding poorly constructed solutions that put self-interested actors in a position to easily defeat the aims of the regulation for their own benefit. Your post reads sort of like a "pick your poison" position with a bias toward market solutions because of their self-correcting nature, but if we are also capable of improving our interventions by learning from our past mistakes and carefully monitoring and punishing undermining behavior, I wonder if that bias might be countered a little bit so that we ought to be just sort of completely neutral in any given case.

    I wonder if I could get you to comment on a case perhaps a bit messier even than lighthouses. I don’t know if you agree with this but it seems to me that American obesity- which Erga makes analogical reference to in the comment above - represents a market failure. The causes of obesity are complicated, but it seems clear that it has a great deal to do with our addiction to highly caloric and inexpensive fast foods. The health and productivity costs associated with obesity seem to vastly outweigh the benefits associated with nourishing ourselves in this way. Is it obvious to you that this is something for which a market solution is preferable to a regulatory one? Perhaps a little bit of both? I really don’t know.

  3. Randy, I think the simple strategy of “recognizing and avoiding” the actions of government bad actors is naively optimistic. Whenever there are centers of power, people work to capture that power for their benefit. They consistently succeed in doing that, in more or less obvious ways. And, again, public choice economists provide a theory about why and how it happens in such a way that the simple strategy is inherently inclined to failure.

    But you and Erga couldn’t have picked a better example of government failure if you were trying to help me make my case. Here’s Micael Pollan on (one aspect of) the obesity problem:

    “the rules of the food game in America are organized in such a way that if you are eating on a budget, the most rational economic strategy is to eat badly–and get fat. This perverse state of affairs is not, as you might think, the inevitable result of the free market. Compared with a bunch of carrots, a package of Twinkies, to take one iconic processed foodlike substance as an example, is a highly complicated, high-tech piece of manufacture, involving no fewer than 39 ingredients, many themselves elaborately manufactured, as well as the packaging and a hefty marketing budget. So how can the supermarket possibly sell a pair of these synthetic cream-filled pseudocakes for less than a bunch of roots? For the answer, you need look no farther than the farm bill. This resolutely unglamorous and head-hurtingly complicated piece of legislation, which comes around roughly every five years and is about to do so again, sets the rules for the American food system–indeed, to a considerable extent, for the world’s food system” (

    So the farm bill is a good place to start looking at what the government has been doing as a direct contributor to the obesity problem. It has also made some attempts to curtail it, but there isn’t any eveidence that these work. For example, where laws have required restaurants to post the calories of menu options, the number of calories purchased have not declined (, Excise taxes on sugary sodas do have a small effect inducing some people to buy less of them, but mostly people without weight problems ( Mostly, the taxes directed consumers towards less expensive sodas or to go for equally sugary non-soda substitutes.

    Of course, many obesity policies have been subject to regulatory capture by entrenched interest groups. Remember when Congress, under pressure from the ag business lobbey, allowed pizza (when it contains a certain amount of tomato paste) to count as a vegetable (

    And, to round out the story, there are the more run-of-the-mill unintended consequences of government intervention. For example, there is clear evidence that Department of Agriculture guidelines on dietary fat contributed to increasing obesity rates (

  4. Kyle, thanks for that. Right, I did pose that example partly because of all the bad regulation that surrounds it, but also because it is not obviously a problem that the market is going to fix. (Though apparently some of the news is good these days You aren’t telling me how that’s going to happen. I don’t think the regulations have actually caused the obesity epidemic, do you?

    I'm just not sure I buy your pessimism about the future of regulation. These folks you like have a theory, so what? It’s based on a data set of regulations that are mostly predicated on a very poor understanding of human behavior and are predicated on extremely crude monitoring and measuring instruments. There are smart folks who disagree with your folks. Do Joseph Stiglitz and Thomas Piketty really just fail to appreciate the power of the points you are making here?

    Of course I agree with you that “whenever there are centers of power people try to capture that power for their own benefit,” -well put- but the fact that people constantly try to do things does not imply that they will constantly succeed. (If it did, then I would point out that wherever there is suffering and injustice people are constantly trying to ameliorate it.) The truth is that within government we have made a great deal of progress in controlling this sort of thing. In the early 20th century government employees in the U.S. collected bribes quite routinely, and this was partly because their pay sucked and they had no job security. Now they don’t typically do this, not because we are hiring better people, but because the jobs are better and the risk associated with collecting bribes is just too great. Why do you suppose we can be successful in this context, but remain doomed to failure in the context of regulating the market?

    The lesson I draw from you is that we need to reject all regulation the success of which depends on the assumption that people will not pursue their own self-interest at the expense of the public good. Good intentions are worth exactly nothing. We need to create conditions in which people generally perceive it to be in their own best interest to simply do the right thing. I also agree that we should favor market solutions when both are on the table, for the reasons you give. But if the market itself looks like it is trying to kill us, or is failing to respond to something that will, I’m still inclined to try to do something.

    What’s interesting to me about the argument you are making is that it doesn’t just apply to market failures; it applies to all government regulation. So, e.g., it applies to progressive taxation policies that attempt to alleviate poverty and somewhat diminish the gap between the rich and the poor. It applies to all environmental regulation. It applies to attempts to ensure the safety of the food supply. Have the most successful countries in the world succeeded in spite of universally creating and employing such institutions?

    1. Randy and Kyle,

      My two cents are much inflated compared to Russell's...

      Does it make a difference whether we apply market or governmental solutions to market failure?

      I was persuaded of the asymmetry by the writings of Mancur Olson (‘The Rise and Decline of Nations’) The concentrated benefits and dispersed costs of government action produces a ‘ratchet’ in government action which is absent in the market. Once a particular government intervention is in place it becomes inordinately difficult to displace.

      The five biggest US corporations in 2010, or order, were Walmart, Exxon Mobil, Berkshire Hathaway, Apple, and General Motors. In 1960 they were General Motors, Exxon Mobil, Ford Motor, and General Electric.

      Significant flux in the private sector.

      And Ford is on the 2010 only list because of a federal bailout in 2009.

      The Mohair Subsidy, designed to ensure a supply of clothing insulation long since surpassed by artificially produced fibers, has (with a brief hiatus) survived from World War II into the current Farm Bill.
      Olson was brought to his reflections by the dynamism of the postwar German and Japanese economies, even after having been bombed flat by the allies in 1945.

      Olson’s own conclusion was a grim one: the only way so far discovered to manage the sclerosis produced by government intervention?

      Lose a war.

    2. The self-correcting nature of the market in contrast to governmental solutions would seem to be in line with a scientific approach to problems. (See Popper)

    3. Thanks, Randy. A bunch of responses (lining up more or less with your paragraphs):
      1. I cited some stuff that makes me think that government regulations have contributed to the obesity epidemic. I consider it a government failure, in the sense (indicated in the OP) that government interventions have made obesity a bigger problem than it would have been had it done nothing. It's difficult to say anything about how the market would fix the obesity problem when it's so difficult to know what the scope of the problem would be absent government interventions.
      2. I'm happy to listen and perhaps respond to some specific criticism of public choice theory (but I couldn't tell whether you were saying that public choice analyses were based on a poor understanding of human behavior/extremely crude measuring?). Anyway, yes, I think Stiglitz in particular fails to appreciate "the knowledge problem" when he makes arguments about the potential positive externalities of appropriately directed industrial policy. Worse, when he presented the financial crisis as a market failure, he failed to acknowledge just how regulated (and captured) financial markets are.
      3. I like the way you put the lesson you draw, but the condition you include matters a great deal when we assess different ways governments intervene. Interventions to address poverty (for example, assistance for needy families, or TANF, vs. a basic income guarantee) or environment problems (EPA regulations vs. cap and trade) can rely on more or less discretionary power on the part of government. I predict you get more government failure when you increase the scope of bureaucratic tinkering, which is arbitrary and discretionary, than when you have government programs that are implemented through simple, abstract and general law. This element -- the rule of law -- is a really good predictor of a country's success.

    4. Thanks Kyle. Regarding paragraph 2, I meant to be saying that the vast majority of government interventions have so far been based on (bad) commonsense reasoning and a poor understanding of human behavior. I would still accept that the burden of proof more or less lies on the interventionist, but I really don't think past failures are evidence of what government regulation can accomplish in the future, anymore than I accept that the effectiveness of medicines developed prior to the 20th century are strong evidence of what medical science could accomplish.

      We are just now barely entering the age of evidence in both medicine and social policy, and we are starting to see some pretty impressive results. People are figuring out how to actually test social policies and implement them on small scales and gradually scale up. I think it's reasonable to suspect that a better understanding of human behavior, better data collection and better systems of punishments and incentives will incrementally improve our ability to do effective regulation, and of course progressive incremental improvement can be transformative.

      And I think we should try very hard to get better at this, for the same reason that we try very hard to get better at medicine. Nature is self-correcting, too, in very much the same way that markets are, but the corrections have an unfortunate tendency to require people we love to die before their time. As Keynes pointed out, markets correct over the long run, and over the long run we all are dead.

      To me the most powerful explanatory reason to be dubious about our ability to do government regulation is not from public choice theory so much as from the far more recalcitrant fact of our very limited ability to accurately predict the behavior of a complex system over more than a small interval. That’s why I think sometimes short term well-informed and very limited tinkerings might ultimately be more effective than the more principled activity you and I tend to prefer.

    5. Randy,

      What do you say to an objection to your thesis about evidence-based piecemeal social engineering (to use Popper’s phrase), based on your analogy with evidence-based medicine.

      The objection has to do with the game-theoretic nature of both.

      However big the Big Data you have as a medical researcher or a government official, you’re still playing a ‘game’ against the world. You’re playing against the microbes on one hand and against the preference orderings of individuals on the other.

      The germs adapt to medical advances because natural selection works. Medical improvements get undermined, unintended consequences abound.

      Producers and consumers will likewise adapt to nudges because they’re intelligent free agents.
      They know where the Twinkies are – behind the carrots.

      What then? This is an honest question.
      The usual responses are to double the dosage of medicine and nudge harder. What would you propose instead?

      This is not a counsel of despair regarding scientific medicine; there we have no alternative. But with piecemeal social engineering we do: the market.

    6. Tom, interesting thoughts. I'm not really sure how game-theoretic medicine is outside the context of infectious disease, and I'm pretty sure we're going to cancel that game more or less completely once we get past the age of antibiotics, which we will pretty shortly. To me, the larger point is that every solution, in every walk of life, brings with it new and unanticipated problems. That doesn’t mean we stop trying to solve the ones we have. First of all, it’s interesting. But secondly, real net good still occurs. I don’t want to go back to driving a horse and buggy just because cars are warming the planet. I want to fix that problem instead.

      In public policy, the problems do come in a game-theoretic form, but I’d say the same thing: just because the solutions create more problems doesn’t mean we have achieved nothing worthwhile.

      You say there is no alternative in medicine, but there is: let nature take its course. It works great! as long as you don't mind people you love dying. We can always make more people. (Parenthetically, as you know, modern medicine hasn’t even particularly extended to average lifespan of humans until very recently. Almost all of that has been due to better sanitation, increasing material prosperity and decreasing social violence.) So to me this comparison remains pretty apt. The reasons we don’t want to just always go with nature’s self-corrections are exactly the same reasons we don’t always want to go with the market’s.

      I absolutely love market solutions and I think we should tend to favor them when they are available. But I also think it’s best not to be hedgehogs with just one-big idea (and ready-made, turn-the-crank, largely aprioristic refutations of other approaches.) We know very well that some regulation is crucial: an unregulated market tends toward exactly the same inefficiencies we see in purely socialist systems, through the monopolistic elimination of competition. All of the corrosive effects Kyle cites can and do occur here as well, but not many serious people argue for the repeal of anti-trust legislation and the FTC. Most of the big regulatory agencies are there to guard against various forms of market failure as well. They don’t do it optimally, but again it seems unlikely that we would be better off without them.

      Ultimately I think the proper synthesis is not doing away with regulation, but harnessing market-based solutions as much as possible when we decide to regulate, as, e.g, with emissions trading. Which Kyle seems to cite approvingly above.

    7. Let's agree to search for a third alternative to:

      1. Ready-made, Turn-the-crank, Largely aprioristic Solutions
      2. Future Scientists of Surpassing Cleverness will Be Able to Do It Exactly Right

    8. I agree. My suggestion is: Future Scientists of Surpassing Cleverness Will Be Able to Aid Us in Making Incremental Improvements Which Over the Long Run are Likely Enough to Be Transformative as to Warrant Cautious Optimism Regarding Our Future Ability to Diminish the Frequency of the Worst Kinds of Market Failures Through Regulation.

    9. Depending on what you mean by 'regulation' I may not disagree. For example, regulations that lead to cronyism emerge from our political institutions that allow easy access for interest groups to political privilege. If you want better political outcomes, you have to examine the institutional rules. Here's one guy of surpassing cleverness doing that:

  5. My two cents here: there are sometimes government pressures that get applied to the people who study a problem (like obesity) in order to make it look like the government's policies are more effective in solving that problem than they really are. For example:

    It will no doubt be countered that such things happen with private corporations too (where the corporation pressures those who study something, to get the result the corporation wants). I do not deny it. But the question then becomes which form of pressure is more common, and more pernicious, and harder to detect when it happens.

  6. Would you consider price controls as interventionist policy?

    Here I am thinking of rent-controlled apartment buildings. While the price ceiling in rent-controlled areas can create a shortage of apartments it can also work against some otherwise unsavory events. If we removed rent-controlled apartments we would see many not-so-wealthy people pushed out of areas where they have lived and established a life in favor of higher rents. An example of this occurred in Venice, CA and Santa Monica, CA. There are two buildings that were built at the same time by the same building company the only difference is the zip code. The building in Venice fetches ~$1,100 a month for a one bedroom, the building in Santa Monica brings in ~$2,000 a month for the same one bedroom. Do you see the lifting of a price ceiling as a positive move? It would move us closer to equilibrium but at the cost of people who depend on having affordable housing.

    1. Thanks for the question, Channing. Rent control and other price controls are definitely interventionist policies. But, are they government failures in the sense of being Pareto inefficient?

      Literally almost all economists say they are (a AER poll showed 93%), since a legally mandated price ceiling causes artificial shortages. Property owners will tend to find ways to supply fewer units when they are required to accept lower prices for them.

      For example, they curtail new investment in affordable housing or convert existing properties to uses that aren’t covered by the policy: a private condo association, short-term vacation rentals or non-residential rentals. Rent control also affects the quality of the supply since owners have less incentive to invest in proper maintenance of properties when returns are low. Some ways the policy causes shortages are even more difficult to detect. For example, renters situated in rent-controlled units will be hard-pressed to give them up, even if they might otherwise have downsized apartments when, say, their grown children have left the home. On the demand side shortages result from lower-than-market prices making renting in the area more attractive to more people, including people who might otherwise buy, rather than rent, or be willing to commute from another area.

      But it might be appropriate to have allocative efficiency take a backseat other social justicey values.

      Unfortunately, the shortages tend to have a disproportionate effect on low-income folks. Since rent-controlled prices no longer function to allocate rental units efficiently, many property owners substitute non-price allocation methods for matching applicants up with available supply. They can be choosy, requiring a super-excellent credit history or forbidding pets or children. Some property owners employ hidden price allocation measures, like demands of illegal “key money.” Economists have shown that rent control tends to have regressive rather than progressive effects. That is, low-income folks tend to do worse under a rent-control regime. Of course, not low-income folks who are lucky enough to land an affordable rent-controlled home, but the other ones.

    2. You raise some very good points against price ceilings which leads to me ask.... what can we do? Is the only option to remove rent-control and let the market decided on housing values?

    3. Yikes, Channing. I'm just a poor philosopher. I have seen arguments that (under certain circumstances) housing voucher programs work better than rent control since they don't interfere with the operation of the price system. They could still distort it, though -- voucher programs can fuel higher rental prices since more people would have more resources to bid them up. There's no getting around the fact that in, e.g., SF, lots of people with lots of money want to live there.

      Is it a market failure that SF is unaffordable for most people? Well, no. You could argue that it's unjust, but a market failure isn't simply a market outcome that people don't like.