Have you ever wondered how every toothpaste company in America can receive the blessings of nine out of ten dentists? Many television commercials used to say, “Nine out of ten dentists recommend XYZ brand of toothpaste.” All the different companies used to say that. Of course, not ALL toothpastes can be recommended by 90% of all dentists. The math does not add up. The trick is that many of the toothpaste companies only asked ten dentists, and nine of them recommended their brands. Never mind the fact that ten other randomly selected dentists might not agree with the first ten, but the irrationally small sample of original ten dentists sure did.
My boss recently pulled some statistic magic out of thin air to brief her boss some rosy numbers. We were making estimates about how many gallons of water people use in a day to drink and do other activities. These numbers were already provided to us broken down by usage type, climate, etc. by a research firm. Originally, we used these highly researched, in-depth numbers in our planning factors. But, the subsequent requirement for bottled water needed for the project was huge. My boss did not like what the numbers were saying, and she knew that her boss would not like them either. So, she changed some of the underlying assumptions and created a more favorable forecast that would be acceptable to the “powers that be”.
Needless to say, the “textbook” researched numbers were actually accurate, and we ran out of water because we based our forecasted usage on our rosy numbers. We eventually had to get an emergency resupply. Our original forecasts were fairly accurate even though they were unpopular. Twisting the statistics and changing the underlying assumptions that were unrealistic proved very costly.
Things like this happen all the time. A friend of mine was recently ecstatic about a report that that was released saying that the Hummer H1, H2 and H3 sport utility vehicles are more energy efficient than a Toyota Prius hybrid and many other similar small vehicles. The study tried to prove that over the life of the car from manufacturing, use, and then disposal, some green cars like the Prius are actually more damaging to the environment than gas guzzlers because of the need for battery disposal, special manufacturing needs, etc. It was a very interesting report and concept that tried to measure environmental impact of the metal usage, recycling, energy usage during the manufacturing process, etc. and compare vehicles’ impact on the environment using metrics more than just solely miles per gallon (mpg).
Many of these calculations are very hard to accurately determine, but the marketing research firm that wrote the report tries valiantly. But, the firm’s assumptions proved to be way off base and faulty. The company that wrote the report used faulty assumptions and biases that swayed the outcome of the research. For example, the company used calculations based the useful life or expect life of the vehicles. They claimed that the Prius for instance has a lifespan of 11years, compared to the Hummer H1’s 35 years. The original study also based its conclusions that the lifetime miles of a Prius were assumed to be 109,000 miles versus 379,000 miles of usage for a Hummer. Not very realistic, is it? But, it is an underlying foundation and assumption that the company bases its statistics and conclusions on. The type of firm alone, a marketing research firm, also lays suspect to the validity of their claims. Who were they ultimately working for, were they biased, etc.?
The old saying still rings true, “Garbage in, garbage out.” You get what you pay for with spotty statistics. And, who pays the consequences? We all do. We are all so bombarded in the Information Age by studies, statistics, samples, census, reports, white papers, and a host of other products from people selling something. And, do not be snowed my friends, everyone is selling something even if it is just an idea. We are now responsible to do our own investigations. We can no longer just accept any statistics that are presented to us because they seem authoritative.
We must always be on guard with the information we allow to sway our decisions and perceptions of reality. The misuse of statistics in investing and personal finance products is alive and well too. So, not only is it “caveat emptor” or buyer beware. Now it is readers, television news watchers, and everyday citizens that must also always be mindful of the misuse and misrepresentation of statistics in our everyday lives.
{ 2 comments }
The only real danger of being manipulated by shoddy numbers is when you don’t bother to be literate with regard to how to consume statistics. If you understand how proper research is conducted and how the statistics are calculated, you’ll immediately discount the vast majority of crappy studies — which is exactly what they are. Statistics are numbers. They can’t lie. But because so many people are afraid of numbers, they become easy marks for the unscrupulous.
Actually, it is possible for 9 out of 10 dentists to recommend all those different brands. You’re just making the assumption that the dentists can only recommend one which isn’t necessarily true. Not to unlike your boss who changed the assumptions, you were working on a different set of assumptions.
As a side note, the company is either leaving things up to chance or being deceptive. If they only ask 10, then as soon as they get a second dentist who woudn’t recommend their brand, their study is over. If they kept asking until they got nine and excluded all but one who wouldn’t recommend their brand, that’s just plain deceptive.
Comments on this entry are closed.