Blog

“Greetings –

Nota bene: I was pretty [upset] when I wrote this, but don’t let my annoyance detract from the message. Bad science is an abomination. Incompetent news reporting on bad science is worse.

You’ve probably heard that breakfast is the most important meal of the day. “What is less commonly mentioned,” writes Alex Mayyasi in The Atlantic, “is the origin of this ode to breakfast: a 1944 marketing campaign launched by General Foods, the manufacturer of Grape Nuts, to sell more cereal.”

Seventy-five years later, here’s the latest report from the April issue of the Journal of the American College of Cardiology: “Taken together, these studies [showing a positive association between skipping breakfast and CVD and CVD risk factors] as well as our findings underscore the importance of eating breakfast as a simple way to promote cardiovascular health and prevent cardiovascular morbidity and mortality.”
What were the findings? Let’s look at a few newspapers:

• “Want to Lower Your Risk for Heart Disease? Eat Breakfast Every Morning” (Healthline)
• “Eating breakfast? Skipping a morning meal has higher risk of heart-related death, study says” (USA TODAY)
• “Study: Skipping breakfast increases risk of heart disease mortality by 87 percent (FOX)”

(You may notice that all three headlines imply causality.)

Looks like General Foods was right. Time to reach for the Lucky Charms? Perhaps it’s time to put on our critical thinking cap instead. The actual study, and the media coverage of it, is a part of the Groundhog Day that is observational epidemiology (for more on the limitations of this type of research, check out Studying Studies: Part II). This was a prospective cohort study pulling data from NHANES III, looking at people who reportedly eat breakfast every day to people who never eat breakfast, and then following up with them (about 19 years later on average), tallying up the deaths from CVD and deaths from all causes.

One question to ask about the population studied is: was eating breakfast or not eating breakfast the only difference between these two groups? In other words, were there any confounding factors (for more on confounding, see Studying Studies: Part IV)? The authors reported that, “participants who never consumed breakfast were more likely to be non-Hispanic black, former smokers, heavy drinkers, unmarried, physically inactive, and with less family income, lower total energy intake, and poorer dietary quality, when compared with those who regularly ate breakfast.” Not only that, “participants who never consumed breakfast were more likely to have obesity, and higher total blood cholesterol level than those who consumed breakfast regularly.” They also had a higher reported incidence of diabetes and dyslipidemia. Read that again, please.

While the study used statistical models to “adjust for” many of these potential confounders, it’s extremely difficult (actually, it’s impossible) to accurately and appropriately adjust for what amounts to fundamentally different people. The healthy user bias (or the inverse, an unhealthy user bias) is virtually impossible to tease out of these studies (the healthy user bias is covered in more depth in Studying Studies: Part I). Not only that, you never really know what you’re not looking for. This is typically referred to as residual confounding in the literature, where other factors may be playing a role that go unmeasured by the investigators.

I haven’t even yet mentioned the misleading nature of reporting relative risk — in this case, an associated 87% (reported in the study as a hazard ratio of 1.87) — without reporting absolute risk. The question you should always ask is, 87% greater than what? To get an idea of the associated absolute risk, the number of CVD deaths in the “every day” breakfast group were 415 out of a total of 3,862 people over 16.7 years (that’s an unadjusted rate of 10.7%) while the numbers for the “never” breakfast folks were 41 CVD deaths out of a total of 336 people over 16.7 years (unadjusted rate of 12.2%). That’s an absolute difference of 1.5% over almost 17 years (annually, this is an absolute difference of 0.09%). Granted, this is before adjustment of the myriad confounders (including the biggest “risk factor” for CVD death, age, in which the “never” breakfast group was younger on average at baseline), but it gives you an idea that we’re looking at small differences even over the course of a couple of decades. This looks a lot difference on paper than an associated 87% increased risk of CVD death. (For more on absolute risk and relative risk, see Studying Studies: Part I.)

There’s more:

• What were the participants actually eating for breakfast? We don’t know. The investigators didn’t have information about what foods and beverages they consumed.
• Did participants change their breakfast eating (or abstaining) habits over the course of almost 20 years? We don’t know. Information on breakfast eating was only collected at baseline.
• Could there be errors in the classification of the causes of death in the participants? It’s possible.
• What constitutes skipping breakfast? Was it the timing of the first meal of the day? We don’t know. Participants were asked, “How often do you eat breakfast?” but there was no definition of what that means, exactly.

What’s more likely: reported skipping breakfast was a marker for a lifestyle and environment that may have predisposed these people to a higher risk of CVD death or that skipping breakfast itself causes CVD death?

Go ahead and skip all the breakfasts you want. And please forward this to the next 10 people who tell you it’s unhealthy to do so.

– … Peter”

For a list of all previous weekly emails, click here.

podcast | website