Does Chocolate Really Help Weight Loss? How Nutrition Headlines Can Be Misleading
Do any of these headlines look familiar to you?
Need a ‘sweeter’ way to lose muffin top? Eat chocolates!
Excellent News: Chocolate Can Help You Lose Weight
Lose 10% More Weight By Eating Chocolate Every Day…No Joke
Chocolate accelerates weight loss: Research claims it lowers cholesterol and aids sleep
Many of these headlines ran on websites as well as in magazines and newspapers a few years back. The information made news in more than 20 countries and six languages. It was all based on a study that was conducted by a little-known researcher named “Johannes Bohannon” from the “Institute of Diet and Health”.
Here’s the problem: both the name of the researcher and the organization were made up. In fact, the entire study was a sham, set up to highlight the problems of taking small snippets of data from poorly-designed research and sharing that information with the public.
According to Mr. Bohannon’s first-hand account, which you can find here, he collaborated with a group from Germany with the eventual goal of releasing a documentary on the project. He changed his name as author of the study from John to “Johannes”, in order to sound more academic. The group created a website for the project called the “Institute of Diet and Health”, which sounded legitimate but could easily have been identified as an empty shell of an organization had anyone cared enough to check.
The study itself consisted of fifteen people divided into three test groups: those who followed a low-carb diet, those who followed a low-carb diet with chocolate, and a control group.
The “research” team did conduct a legitimate study in the sense that they had participants follow consistent protocol and then collected legitimate data. Some of what made the study “bad science”, however, included:
The small sample size. It’s notoriously misleading to take results from a very small group of people and apply them to a larger population.
The researchers didn’t have a hypothesis, or prediction, for the study. Instead, they measured eighteen different variables, including weight, cholesterol, sodium levels, sleep quality, and more. This is called “p-hacking” in the scientific community, but it’s comparable to buying lots of lottery tickets to improve your odds of winning a jackpot. In this instance, the “researchers” needed only one variable to “hit” in order to produce an eye-catching headline attributing a health benefit to chocolate. That variable happened to be weight loss.
The change in weight was technically statistically significant, but the change was small (five pounds) and the study was very short (three weeks). Measuring lots of different things in a small group of people for a short period of time helped guarantee that the group would find a meaningful change in something. In a long-term study, that change would likely just appear as an outlier within a much larger set of data.
Eventually, the study’s authors revealed that the study was meant to be fake and misleading. Some news outlets published retractions and corrections. Others, two-plus years later, still have their original articles posted.
Maybe it’s not surprising that media outlets picked up this story and published it, but there’s no way this farce of a study was published in an academic research journal, right?
Actually, it was.
Why should you care about this? This study happened to be intentionally misleading, but it’s common practice for media outlets to publish headlines that don’t accurately represent what good science has shown. Individual foods are often portrayed to have significant effects on the body, but that’s not the case when put into real-life practice.
Want another example? There are plenty. Cayenne pepper “burns fat and boosts metabolism”! Well, maybe a little bit. The reality is you’d need to consume 5.5 teaspoons of cayenne pepper per day to have any meaningful effect – if you’ve ever consumed more than half a teaspoon of cayenne pepper at one time you know how unrealistic that is. More reasonable amounts of the spice are only going to burn around ten calories or fewer in a day’s time. Not exactly a game-changer, huh?
If you’re comfortable reading scientific literature – and have plenty of free time – read these studies for yourself if the information seems too good to be true. If not, a dietitian is a great resource to give you proper perspective on what individual foods really do – and don’t do – in your body.