Saturday, February 27, 2010

Surveys, statistics and statistically significant economic tremors

Once in a while, an article would pop up in the news and make me go, "oh great, here comes another guy who is talking about statistics, but knows nothing about it". This article on the BBC made me feel just like that, but luckily only in the first half: How one woman can cause economic boom or bust. However, having finished reading it, I came to appreciate his point. He is portraying how the world, especially when in crisis like these days, is reacting to 0.1% changes in unemployment rate or deviation from economic forecasts, without fully understanding the data source the conclusions are drawn from, or the statistical significance level it can be trusted to.

The author goes quite the distance to move his reader's emotions, and raise my suspicion:

She (the lady in the fictitious story who just lost her job and by chance was surveyed by the Labour Force Survey) is just one of those surveyed. But Eve, unknowingly, is about to move mountains. She will make economies tremble with a 30-minute interview and a cross in a box on a laptop questionnaire.

Vast sums of money will lurch round the world's financial system. Politicians will reel and businesses be broken.


But then he comes back across the line and is in my good books again:
Check the ONS (Office for National Statistics - UK) and it states clearly that the figure is accurate only to 0.2 per cent, most of the time. This means that a rise of 0.1 per cent in the unemployment rate could be consistent with an actual fall in unemployment across the whole economy of 0.1 per cent.

I like his final point the best, suggesting how people should treat survey results - more like clues, not knee jerk reactions to trigger panics:

... feverish times make attention twitchy. Every piece of evidence about the state of the economy is interpreted, explanations offered, forecasts recalculated, and much is made out of little, perhaps too much.

The difference between a rise and a fall is judged with solemn faces when the truth is the change we observe may not even be there. Economic data is never a set of facts; it is a set of clues, some of which are the red herrings of unavoidable measurement error.

No comments: