Coincidence, correlation and chance

Stumbled across this. It's one of those great articles which explain why you can't always trust statistics.

I particularly liked the first section on coincidence: the way we notice and are amazed by things which seem extraordinarily unlikely, whether events or patterns. And then often jump to very wrong conclusions by trying to explain them away.

But as the author points out, these coincidences are rarely as amazing as we might think. Particularly with events, sometimes they are not coincidences at all. It's just that we rarely notice the 'near misses' that are maybe happening round us all the time. Statistically speaking, they are no big deal. It's all just a function of big numbers - the number of times these extraordinary events COULD happen to people multiplied by the number of people it could happen to. On this basis, no matter how unlikely it is almost certain to happen...to someone. Scary.

And at the other end of the spectrum, there are those coincidental patterns which ARE completely random (again because of big numbers), but where we force fit meaning. This is because our brains are hardwired over Millennia of evolution to find patterns and meaning in things (the old 'is it a stick; is it a snake' thing, where instinct kicks in and says jump). So we still look for, and often invent, explanations for the inexplicable. There's a great example of how you could persuade people to part with big sums of money by seemingly predicting football results.

Anyway, well worth a read. And a salutary lesson for when we're poring over U&As, Nielsen reports or tracking studies trying to find meaning - the amazingly insightful might well be. But it could also be completely common place (we just haven't noticed) or completely wrong (because we've intuited patterns which just aren't there). So tread with care!