Here’s a bold statement, “Dirty up your act!”
I began to think about statistical analysis the other day and something one of my professors in Probability and Statistics said made him crazy, “Users of creative analysis, should not be licensed to carry calculators.” It’s the idea that a set of data, say the amount of apples cancer patients eat in a year can predict the health benefits of apples for the rest of us, can be very enticing. Finding out that this particular set of patients did not eat an entire bushel of apples every year, “unmistakably” leads us to the conclusion that they failed to protect themselves with that obvious anti-cancer wonder drug. The problem with this kind of analysis is that we want it to predict that there are anti-cancer benefits to apples, and so it does.
Apples are nice, fresh (well, most anyway) and seem to scream “Health!” It seems just a hop, skip and a few taps of the array key to conclusive proof that apples are the anti-cancer weapon of the future. The stray facts that 60% of the study’s participants were addicted to dental X-Rays and the other 40% drank a dose of motor oil every day can be left pleasantly aside.
A great deal of “healthy living” advice is based on statistical analysis, or as I always suspect “creative analysis.” We have become a nation of dutiful hand washers and vegetable eaters based on “scientific” analysis of health benefits from these noble pastimes. And yet, quite mysteriously, cancer rates continue to increase, new and terrible infections are on the rise and there is a veritable epidemic of the need for anti-anxiety drugs. I often wonder if the last is due to the failure of our current analyses to offer significant protection from the rest.
What started me on this line of thought, was the recent death of my sister from cancer. She lived as about a healthy life as one could, resisting all forms temptation, working hard, sleeping well and remaining svelte her entire life. And then I thought of my own first wife who gave up drinking, smoking and began to carefully watch her weight when we first had children, and then died of breast cancer shortly after out third son was born. Or my brother in law who also gave up smoking, and severely limited his drinking about 15 years before he died of cancer. Does that mean anything? My professor would say the population sample is pitiful, and there was no possibility of significant factor development. Ok, so “clean living” didn’t kill them?
He’d be right about significant factors, of course. But continuing to press along this line of non-sensical reasoning let’s consider the human immune system. It’s a very clever design. Our immune systems function so well, not because they act like defenders behind impenetrable walls, but rather as Jonas Salk might say “because they let the bad guys in.” It’s a system that does essentially the seemingly worst possible thing for the body it protects. However, Earth is a planet where it’s impossible to maintain complete sterility. The air, land,waters and every surface are teeming with organisms of every shape and size. We can scrub and shower all day every day and we are still covered in germs, viruses and other microorganisms from head to toe. Being well developed for Earth dwellers, our immune system welcomes these pesky invaders into the inner sanctum and then learns how best to destroy them. Because it doesn’t forget an enemy, it’s even more ready the next time. So maybe we shouldn’t wash our hands, for fear of dumbing down our immune system?
However, subject that same amazing immune system to “creative analysis” and we would be soon be temped to scrap the whole works. Without much math we probably can agree that just about everyone does get sick from time to time. With 100% chance of getting sick sooner or later that failure rate looks pretty dismal, especially because we just can’t tell how many sicknesses were successfully fought off. The “creative analyzer” may well assume zero diseases defeated. Who could prove him wrong?
Which makes daily hospital-level sanitary precautions for everyone seem much less loony and has made hand sanitizers a best selling product. The fact that we can’t tell if these sanitizers have any effect at all, and may well be the worst thing we could do, appears more like a protection value instead of a failure rate. What is the cumulative effect of hand sanitizers, are they a viable alternative to a well tuned immune system? Or is it better to fight disease on the inside? That’s certainly not what the makers of hand sanitizers want to hear.
But nice, clean and scented hands are attractive, surely they belong in the “halls of health.” Consequently children should never play in the dirt, people with diseases should be shunned and “Why don’t they sell hand-sanitizer in 55 gallon drums with wheels on the bottom?”
Statistics “prove” it all. However, as someone who did pretty well in statistical and probability theory let me mention the familiar and well informed phrase from Benjamin Disraeli, “There are three kinds of lies: lies, damned lies, and statistics.”
One of the main problems with trusting statistical analysis is only the best studies include three crucial pieces of information; the “angle”, the “population” and the “standard of deviation.” I have little doubt that most times these items of gobbledy-gook and mathematical michigas are invariably left filed in the back room of far away lands for the sake of “clarity.”
In this case the “angle” is the statisticians approach to the mass of raw data. Every study has an angle, it’s both what made them look in the first place and the method they used. In the anti-cancer apple case above, maybe the cause for the study was the stunning discovery of a possible link between cancer and the fruit, or maybe it was a grant from the Apple Growers of Oregon. Either would have an appreciable effect on the method as well, wouldn’t it? After all, which is more useful to the AGO, the amount of apples they did eat (god forbid!) or the ones they didn’t?
The population is important as well. Was this a study of 2000 swarthy, balding, Croat asbestos handlers, or 20 elderly ladies from Chernobyl with rubber false teeth? Even knowing the details of the population, for the most part it’s possible to prove just about any assertion with any population depending on the data you collect. As in the same case above, “Goodness me, why would we ask about dental X-rays?” The best fights over statistical analyses are always about what the study didn’t look at.
If there is one piece of information that should be included in every study it’s the Standard of Deviation. Whether it’s “70% of doctors know that Camel filters provide the smoothest smoke” (the dead ones, anyway) or “90% of choosey mothers chose Jif” (sample taken in the town of Choosey, Wisconsin) the Standard of Deviation is a kind of average of the answers that disagree with the results of the study. The standard of deviation can tell us that as many as 2 out of 5 candidates did eat a bushel of apples and got cancer anyway. But not always, “What about this guy who ate three bushels of apples every month for his entire life?”, “Can’t you read? Those were Pipins! They’re grown in Washington.”
Copyright Prentiss Gray 2011