Statistical Evidence

The other day there was another medical announcement which indicated that perhaps fat wasn’t so bad after all — seems it is essential for brain development. My wife and I have almost stopped reading these things — whatever is bad for you this week will likely be determined to be good for you next week. We are continually amused by the foods that fall in and out of favour, almost worse than skirt lengths and hairstyles.

And following the recent US elections there has been a spate of columns arguing for or against political campaigns that were based on ‘big data’ — statistical analysis of voters, their preferences, sensitivities and associated variables. Should the candidate part their hair on the right? How many votes will this pick up or loose? Ad nausea.

And medicine argues that by associating numbers with anecdotes about patient conditions and treatment responses they have become ‘evidence-based’. And treatments based on statistics are more valid than the old anecdotal approach — even though the underlying physiological mechanisms are no more understood than before.

There are, of course, the endless streams of meta-analysis where someone has collected the results of a whole population of studies and by suitable manipulation and data filtering declared that the real results of those studies were different than previously claimed.

Recently those of us on Amherst Island were treated to another example of this approach. Health Canada, assisted by a number of wind industry folks and fellow travellers (love that phrase from the 1950’s that loosely suggests unindited co-conspirators). No peer review, no involvement in the process of the protest groups or people who were not disbelievers. The media trumpeted that there were no health issues found. The report was a bit more nuanced — they did not talk to anyone who abandoned their homes. Those who were still there having problems (which were called ‘annoyances’) and found to have elevated stress hormones were told that this was due to being annoyed. So we had a case where ‘A’ (wind turbine noise) => ‘B’ (being annoyed by it) was somehow divorced from ‘C’ (biological markers of stress) but they did find that the people who were having problems did have the biological markers. My head hurts.

The global problem with all of this is that various attributes about the real world are sampled and subject to some analytical massaging. Then the proponent announces that the samples collected allow them to declare that the world works in a certain way and based on their study they could be sure that if certain things were done there would be specific desirable results. But no biological mechanism. I guess in a way what is happening is that various markers for a path through the unknown wilderness have been identified and marked on a map. These markers are connected to form a map — then the map is declared to be the territory. And confusion results.

I guess it is just very human to look for shorthand frameworks for explaining a large and complicated world. But it troubles me that there is so little humility about the extent of our knowlege. It is as though the classic phrase ‘only fools are absolutely certain of their facts’ were a broad and general statement about humanity as a whole. I hope not but it is hard to ignore.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s