Statistics are persuasive… So much so, that people, organizations, and whole countries base some of their most important decisions on organized data. But there’s a problem with that. Any set of statistics might have something lurking inside it; something that can turn the results completely upside down.
For example, imagine that you need to choose between two hospitals for an elderly relative’s surgery. Out of each hospital’s last 1000 patient’s, 900 survived at Hospital A, while only 800 survived at Hospital B. So, it looks like Hospital A is the better choice. But, before you make your decision, remember that not all patients arrive at the hospital with the same health level.
And, if we divide each hospital’s last 1000 patients into those who arrived in good health and those who arrived in poor health, then the picture starts to look very different. Hospital A had only 100 patients who arrived in poor health, of which 30 survived. But Hospital B had 400,and they were able to save 210. So, Hospital B is the better choice for patients who arrive at the hospital in poor health, with a survival rate of 52 %.
And, what if your relative’s health is good when he/she arrives at the hospital? Strangely enough, Hospital B is still the better choice, with a survival rate of over 98%. So, how can Hospital A have a better overall survival rate, if Hospital B has better survival rates for patients in each of the two groups? What we’ve stumbled upon is a case of Simpson’s paradox, where the same set of data can appear to show opposite trends, depending on how it’s grouped. This often occurs when aggregated data hides a conditional variable, sometimes known as a lurking variable, which is a hidden additional factor that significantly influences results.
Here, the hidden factor is the relative proportion of patients who arrive in good or poor health. Simpson’s paradox isn’t just an hypothetical scenario. It pops up from time to time in the real world, sometimes in important contexts. One study in the UK appeared to show that smokers had a higher survival rate than nonsmokers, over a twenty-year time period.