Clearing Away the Rubbish

“The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue”
~Richard Horton, editor in chief of The Lancet

“It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgment of trusted physicians or authoritative medical guidelines. I take no pleasure in this conclusion, which I reached slowly and reluctantly over my two decades as an editor”
~Dr. Marcia Angell, former editor in chief of NEJM

Back in September, there was a scientific paper published in Clinical Cardiology, a peer reviewed medical journal that is “an official journal of the American Society for Preventive Cardiology” (Wikipedia). It got a ton of attention from news media, social media, and the blogosphere. The reason for all the attention is that, in the conclusion, the authors claimed that low-carb diets had proven the least healthy over a year period:

“One-year lowered-carbohydrate diet significantly increases cardiovascular risks, while a low-to-moderate-fat diet significantly reduces cardiovascular risk factors. Vegan diets were intermediate. Lowered-carbohydrate dieters were least inclined to continue dieting after conclusion of the study. Reductions in coronary blood flow reversed with appropriate dietary intervention. The major dietary effect on atherosclerotic coronary artery disease is inflammation and not weight loss.”

It has recently been retracted and it has come out that the lead author, Richard M. Fleming, has a long history of fraud going back to 2002 with two FBI convictions of fraud in 2009, following his self-confession. He has also since been debarred by the U.S. Food and Drug Administration. (But his closest brush with fame or infamy was his leaking the medical records of Dr. Robert Atkins, a leak that was behind a smear campaign.) As for his co-authors: “Three of the authors work at Fleming’s medical imaging company in California, one is a deceased psychologist from Iowa, another is a pediatric nutritionist from New York and one is a Kellogg’s employee from Illinois. How this group was able to run a 12-month diet trial in 120 subjects is something of a mystery” (George Henderson). Even before the retraction, many wondered how it ever passed peer-review considering the low quality of the study: “This study has so many methodological holes in it that it has no real value.” (Low Carb Studies BLOG).

But of course, none of that has been reported as widely as the paper originally was. So, most people who read about it still assume it is valid evidence. This is related to the replication crisis, as even researchers are often unaware of retractions, that is when journals will allow retractions to be published at all, something they are reluctant to do because it delegitimizes their authority. So, a lot of low quality or in some cases deceptive research goes unchallenged and unverified, neither confirmed nor disconfirmed. It’s rare when any study falls under the scrutiny of replication. If not for the lead author’s criminal background in the Fleming case, this probably would have been another paper that could have slipped past and been forgotten or else, without replication, repeatedly cited in future research. As such, bad research builds on bad research, creating the appearance of mounting evidence, but in reality it is a house of cards (consider the takedown of Ancel Keys and gang in the work by numerous authors: Gary Taubes’ Good Calories, Bad Calories; Nina Tiecholz’s The Big Fat Surprise; Sally Fallon Morrell’s Nourishing Diets; et cetera).

This is why the systemic problem and failure is referred to as a crisis. Fairly or unfairly, the legitimacy of entire fields of science are being questioned. Even scientists no longer are certain which research is valid or not. The few attempts at determining the seriousness of the situation by replicating studies has found a surprisingly low replication rate. And this problem is worse in the medical field than in many other fields, partly because of the kind of funding involved and more importantly because of how few doctors are educated in statistics or trained in research methodology. It is even worse with nutrition, as the average doctor gets about half the questions wrong when asked about this topic, and keep in mind that so much of the nutritional research is done by doctors. An example of problematic dietary study is that of Dr. Fleming himself. We’d be better off letting physicists and geologists do nutritional research.

There is more than a half century of research that conventional medical and dietary opinions are based upon. In some major cases, re-analysis of data has shown completely opposite conclusions. For example, the most famous study by Ancel Keys blamed saturated fat for heart disease, while recent reappraisal has shown the data actually shows a stronger link to sugar as the culprit. Meanwhile, no study has ever directly linked saturated fat to heart disease. The confusion has come because, in the Standard American Diet (SAD), saturated fat and sugar have been conflated in the population under study. Yet, even in cases like that of Keys when we now know what the data shows, Keys’ original misleading conclusions are still referenced as authoritative.

The only time this crisis comes to attention is when the researcher gets attention. If Keys wasn’t famous and Fleming wasn’t criminal, no one would have bothered with their research. Lots of research gets continually cited without much thought, as the authority of research accumulates over time by being cited which encourages further citation. It’s similar to how legal precedents can get set, even when the initial precedent was intentionally misinterpreted for that very purpose.

To dig through the original data, assuming it is available and one knows where to find it, is more work than most are willing to do. There is no glory or praise to be gained in doing it, nor will it promote one’s career or profit one’s bank account. If anything, there are plenty of disincentives in place, as academic careers in science are dependent on original research. Furthermore, private researchers working in corporations, for obvious reasons, tend to be even less open about their data and that makes scrutiny even more difficult. If a company found their own research didn’t replicate, they would be the last in line to announce it to the world and instead would likely bury it where it never would be found.

There is no system put into place to guard against the flaws of the system itself. And the news media is in an almost continual state of failure when it comes to scientific reporting. The crisis has been stewing for decades, occasionally being mentioned, but mostly suppressed, until now when it has gotten so bad as to be undeniable. The internet has created alternative flows of information and so much of the scrutiny, delayed for too long, is now coming from below. If this had happened at an earlier time, Fleming might have gotten away with it. But times have changed. And in crisis, there is opportunity or at very least there is hope for open debate. So bring on the debate, just as soon as we clear away some of the rubbish.

* * *

Retracted: Long‐term health effects of the three major diets under self‐management with advice, yields high adherence and equal weight loss, but very different long‐term cardiovascular health effects as measured by myocardial perfusion imaging and specific markers of inflammatory coronary artery disease

The above article, published online on 27 September 2018 in Wiley Online Library (, has been withdrawn by agreement between the journal Editor in Chief, A. John Camm and Wiley Periodicals, Inc. The article has been withdrawn due to concerns with data integrity and an undisclosed conflict of interest by the lead author.

A convicted felon writes a paper on hotly debated diets. What could go wrong?
by Ivan Oransky, Retraction Watch

Pro-tip for journals and publishers: When you decide to publish a paper about a subject — say, diets — that you know will draw a great deal of scrutiny from vocal proponents of alternatives, make sure it’s as close to airtight as possible.

And in the event that the paper turns out not to be so airtight, write a retraction notice that’s not vague and useless.

Oh, and make sure the lead author of said study isn’t a convicted felon who pleaded guilty to healthcare fraud.

“If only we were describing a hypothetical.

On second thought: A man of many talents — with a spotty scientific record
by Adam Marcus, Boston Globe

Richard M. Fleming may be a man of many talents, but his record as a scientist has been spotty. Fleming, who bills himself on Twitter as “PhD, MD, JD AND NOW Actor-Singer!!!”, was a co-author of short-lived paper in the journal Clinical Cardiology purporting to find health benefits from a diet with low or modest amounts of fat. The paper came out in late September — just a day before the Food and Drug Administration banned Fleming from participating in any drug studies. Why? Two prior convictions for fraud in 2009.

It didn’t take long for others to begin poking holes in the new article. One researcher found multiple errors in the data and noted that the study evidently had been completed in 2002. The journal ultimately retracted the article, citing “concerns with data integrity and an undisclosed conflict of interest by the lead author.” But Fleming, who objected to the retraction, persevered. On Nov. 5, he republished the study in another journal — proving that grit, determination, and a receptive publisher are more important than a spotless resume.

2 thoughts on “Clearing Away the Rubbish

  1. A response to a post at another blog:

    “The causes of this anger come easily to mind; loved ones lost through the denial of adequate medical care caused by the pursuit of quack remedies, and the crushing despair that follows false hope… The people who promote the claims we skeptics oppose sometimes get angry as well, but with much less real moral justification – they are angry because skeptics are… reducing their clientele for whatever untested or failed “alternative” medical modalities they promote.”

    There is that. My grandmother died from cancer. It’s uncertain what might have been the outcome had it been treated, but we’ll never know. She instead chose to do a macrobiotic diet and new age practices, specifically Science of Mind “mind treatments”. My dad talks about how much faith she had that she could overcome her illness and how the failure put her in a bad place for the brief time she had left. That demonstrates what you’re talking about. On the other hand, the mainstream medical field has had many areas of failure that have caused many to mistrust it, often because lack of skepticism among defenders of the status quo or else defenders of what is profitable. Consider antidepressants that have been pushed for longer than I’ve been alive, but recent meta-analysis shows that they are unhelpful for most people they are being prescribed to (and with serious side effects that can actually harm health). There is immense money to be made in doling out drugs to the mentally ill. Other methods of treatment aren’t nearly as profitable because they are not as easily made into a marketable product. There has been much reporting on the influence of pharmaceutical lobbyists and representatives.

    Diet and nutrition is another contested area. One survey showed that most doctors lacked basic knowledge about nutrition. That is unsurprising as it hasn’t been a priority in medical schools. Yet doctors regularly act as experts on nutrition and many people trust their supposed expertise. The problem goes much deeper than this, though. The official dietary recommendations was largely based on speculation and low-quality research that came to be treated as fact, even though there was other research at the time that pointed to entirely different conclusions. There is a number of contributing factors. For some decades now, it has been known that a replication crisis is plaguing diverse fields of scientific research. Nutrition research has been a major victim of this scientific failure, not that it is alone in this. The problem for this field is that much of the research is done by doctors, but as with education in nutrition most doctors never received much education in statistics and research methodology. Few doctors are trained researchers and so the field is rife with badly designed studies and poorly analyzed data.

    This has allowed for particular dietary ideologies to dominate for the past few generations. Some insightful and detailed science journalism about this has been done by Nina Teicholz and Gary Taubes. What went wrong with this field goes beyond mere strong opinions. There were some charismatic and influential figures in the field, e.g., Ancel Keys, who came to have outsized influence over public debate and public policy. Many critics, even when they had the science on their side, were silenced or became excluded, such as not being invited to speak at scientific conferences, not being able to get funding for research, and not being published in the best journals. This failure was made possible partly because of the limitation of early nutrition research. It heavily depended on epidemiological studies which look for associations but can’t easily control for confounding factors. So, for example, Ancel Keys began with the hypothesis that saturated fat was to blame for heart disease, went looking for information that confirmed his beliefs while ignoring what contradicted it, but even then his own data (as later re-analyzed) always had shown that sugar was the stronger correlation than saturated fat. Nutrition research still largely relies on this weak form of evidence that easily falls prey to bias.

    It’s not limited to dietary issues either, but I would point to another area that overlaps with nutrition research. There has been a contentious debate over statins, involving side effects and overprescription. After decades of the pro-statin camp controlling the conventional views in medicine, two respected peer-reviewed British medical Journals, the Lancet and BMJ, are now in the middle of a fight over this issue. Teicholz was drawn into it when she published an article in the BMJ. The disagreement is not only about statins themselves but also about what we scientifically know about cholesterol and its role in health. Because of the replication crisis, there has been much muddying of the water. In 2005, the standards for research were improved and that seems to have been the turning point, as most studies since then have supported the position of the statin skeptics. Yet the older and more well-established research, although lower quality, continues to hold powerful sway in the research community.

    On a positive note, after about a half century of suppression of alternative views within these areas of research, lively debate has been reawakened. A large part of this had to do with the internet which has given a platform for dissenting and skeptical voices. It’s much more difficult to silence scientific researchers and academic experts who refuse to toe the line. Also, the evidence in support of the alternative views keeps accumulating and has come to a point where it is harder to ignore, even though much of the evidence has been around since the early 20th century and in some cases earlier than that. An example of this is the ketogenic diet. It was originally researched for epileptic seizures back in the 1920s. Then studies looked into its use for diabetes shortly after that. But when medications were discovered and marketed for both conditions, the funding for research dried up. This is in spite of the fact that the medications were never as effective in comparison. To this day, the ketogenic diet remains one of the most effective treatments for both of these conditions, but there simply is no profit to be made in it. What helped turn the tide was when a Hollywood filmmaker got involved because of his son’s epilepsy. He discovered the ketogenic diet in the medical literature and, after successfully treating his son, started the Charlie Foundation in 1994. His advocacy jumpstarted ketogenic research after decades of barely getting any attention. It has since grown into a major area of medical knowledge, having been studied with numerous physical and mental conditions.

    The challenge is that no one has sole rights to skepticism. Those defending the status quo often claim to be the skeptics and they simultaneously often have the most influence over public perception of how views are perceived. The most skeptical voices sometimes get portrayed as the complete opposite for anyone who advocates an alternative view is easily painted as an ideologue, whereas those defending mainstream ideology are of course merely being reasonable or that is how they perceive themselves. To understand genuine skepticism requires taking the long view. Also, it’s important that we be skeptical, even of our own self-identified skepticism. Skepticism isn’t a single position but a contested battleground. And the fight isn’t always fair. Future generations will probably be better judges of this.

Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s