When asking the question changes the answer

Some economics is about trying to solve economic and social problems. Some economics is about trying to work out why those problems occur. And a lot of the rest is trying to work out how to measure those problems and their effects.

Oh, the problems with measuring things. Nothing we can properly measure in the real world seems to exactly correspond to the economic theory we’re taught on degree courses. And then once you’ve measured something you’re not sure which way causality runs. Measurement problems have a whole literature of their own, some of which you will find in econometric textbooks, but is too much to go into in a post on a blog that is, after all, called “Five Minute Economist’s Blog”. I should at least try to be consistent.

But what if trying to measure things changes what you’re trying to measure? Goodhart’s Law, named after economist Charles Goodhart, states that once you try to target a particular indicator for policy, the measure you are using ceases to be useful. So for example, a government might try to reduce internet piracy by targeting the number of illegal downloads (let us, for the moment assume that the government can in fact do this, say by remotely monitoring illegal download websites). Then it might well find that illegal downloading has fallen, but that everyone has migrated onto websites where pirated content is streamed instead. Obviously, Goodhart’s Law doesn’t always hold. But the word “law” in economics tends to be used in vague-not-actually-a-law-more-a-sort-of-observation-that-may-or-may-not-be-true-sort-of-way. We’re not talking laws of gravity type laws here.

However, it doesn’t just end there: just trying to measure something can change the answer. Particularly when it comes to measuring people’s preferences or actions. Because people just don’t behave the way they are supposed to in economic models.

A recent paper from German economists Lora Todorova, Siegfried Berninghaus and Bodo Vogt shows exactly this. They run experiments looking at how risk-averse people are when playing a particular game. And they find that people behave differently if, before they play the game, they are asked questions about their risk preferences. Specifically, people play in a more risk averse way if they have answered a questionnaire first. And, according to the researchers, this happens even if the questionnaire is completely neutral (i.e. no leading questions or wording that would prime people to think in a risk-averse way).

Yet another example of how economics seems so perfect when you learn the theory, before you actually have to apply it.

The full working paper, “A simple questionnaire can change everything” is here.

This entry was posted in New research, Thoughtbox and tagged , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s