Ep. #619: Good Science, Bad Science: The Difference and Why It Matters

Hello, and welcome to another episode of Muscle For Life. I’m Mike Matthews. Thank you for joining me today. And this episode is special because it is one of the chapters of my newest book, newest audio book in this case, but it’s also available in. Digital and hard copy formats as well. And it is called Fitness Science Explained, and you can get it right [email protected].

And this book is a crash course in reading, understanding and applying scientific research. And it teaches you in very simple terms, what most people will never know about how to use science to optimize your. Fitness and lifestyle. Fitness. Science explained covers all of the big moving parts, including the basics of the scientific method, the differences between randomized trials and observational studies.

The power of the placebo effect, the importance of sample sizes, the anatomy of statistical analysis, and much. You’ll also learn in the book how to get access to full text studies without spending a fortune and the most popular journals out there for exercise, nutrition and supplementation. And you will get a scientist formulated cheat sheet that will help you quickly and accurately estimate the quality of research that you want to review.

In my opinion, the cheat. Alone is worth the cost of the book. It is very practical. So whether you want to discover and use evidence-based methods for building muscle or losing fat faster, or maybe reducing your risk of disease or dysfunction, or just maximizing some other aspect of your body, mind, or life, this book will show you the.

Also to celebrate to this joyous occasion, I am giving away $1,500 in Legion gift cards, and all you have to do for a chance to win is head over to fitness science book.com by a copy of the book, any format, and then forward your receipt. Email to [email protected] and that’s it. You are entered in the giveaway.

You gotta act fast though, because winter is coming. No, because winners will be chosen this Friday, September 4th. Oh. And you can also increase your chances of winning by buying. Extra copies of the book, again, any formats. So specifically if you buy three copies, you’re gonna get five giveaway entries. So that is a plus 400% chance to win.

If you buy five copies, you’re gonna get eight giveaway entries, and that is a plus 700% chance to win. And if you buy 10 copies, you’re gonna get 15 giveaway entries, which is a plus 1400% chance to. And you are going to get an autographed copy of the book. So for instance, if you buy the paperback ebook and audiobook, that is three copies, five entries to win.

And if you were to buy three paperbacks as well as the ebook and audiobook, that’s five. So you get. Eight entries to win and so forth. Anyway, to get your copy or copies of Fitness Science explained, just head over to fitness science book.com. Now also, if you like what I am doing here on the podcast and elsewhere, definitely check out my sports nutrition company Legion, which thanks to the support of many people like you, is the leading brand of all natural sports supplements in the.

We’re on top because every ingredient and dose in every product is backed by peer-reviewed scientific research. Every formulation is 100% transparent. There are no proprietary blends, for example, and everything is naturally sweetened and flavored. So that means no artificial sweeteners, no artificial food dies, which may not be as dangerous as some people would have You.

But there is good evidence to suggest that having many servings of artificial sweeteners, in particular every day for long periods of time may not be the best for your health. So while you don’t need pills, powders, and potions to get into great shape, and frankly, most of them are virtually. Useless.

There are natural ingredients that can help you lose fat, build muscle, and get healthy faster, and you will find the best of them in legion’s products to check out everything we have to offer, including protein powders and protein bars pre-workout. Post workout supplements, fat burners, multivitamins, joint support, and more.

Head over to www.by legion.com, bu y legion.com, and just to show how much I appreciate my podcast peeps, use the coupon code M F L at checkout and you will save 20% on your entire first order. So again, if you appreciate my work and if you wanna see more of it, and if you also want all natural.

Evidence-based supplements that work. Please do consider supporting Legion so I can keep doing what I love, like producing more podcasts like this Chapter one. Good Science, Bad Science, The Difference and Why it Matters. The saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom.

Isaac Asam. One month media headlines blazing. That research has confirmed that one food or another reduces your risk of cancer, diabetes, obesity, or some other nasty health condition. Huray, you think time to load up on said food. Then sometime later, after it has become a staple in your meal plans, the other shoe drops.

New research, refutes earlier findings and demonstrates that it actually increases your risk of disease and dys. What the heck? How can scientific research just turn on a dime like that and do a full 180? Oh you think a few months of eating this way can’t have been that harmful Life goes on. Then it happens again and again.

Eventually you conclude that science can’t seem to make up its mind on anything, and you stop paying a. Fortunately this isn’t true. It may appear that there’s a study to prove or disprove just about any assertion, but this illusion isn’t the fault of science itself, but rather the widespread misunderstandings about the scientific process, media sensationalism, and sometimes even fraudulent research.

Let’s take a closer look at the nine main reasons that science can appear to be so confusing and contradict. One media misrepresentation. Attention spans are shorter than ever these days, and when news organizations have just a few hundred words or seconds to report on health matters, they can’t afford to discuss the nuances of complicated scientific research.

Instead, they need titillating headlines and easily digested sound bites that draw eyeballs and clicks and bounce around in social media and water cooler conversations that inevitably leads to misinformation. The two most common ways this occurs are one, confusing correlation with causation. Two oversimplification and sensationalism.

Let’s go over each confusing correlation with causation. Quite a bit of health related research is based on observational data, meaning that scientists observe groups of people going about their lives, collect various types of data, and then look for correlations between different variables. Correlation is a mutual relationship or connection between two or more things.

For example, it was through observational research that the link between smoking and lung cancer was first discovered in the famous British doctors study of 1954, scientists sent out questionnaires to British doctors asking them about their smoking habits. The scientists then looked at which doctors got lung cancer and found that doctors who reported smoking were more likely to get the disease.

This type of research is a fantastic tool for documenting phenomena, forming hypotheses, and pointing the way for further research. But it can never be used to conclusively determine the cause of the phenomena observed, because there are many ways for variables to be related without one ly influencing the other.

For instance, ice cream intake goes up in the summer, as does the incidents of drowning. So you could say that there’s a strong correlation between eating ice cream and. This does not mean that eating ice cream causes people to drown, however, which is how your average media outlet might explain it. A good example of this is how the media has reported that drinking diet soda can make you fat, cause and effect cut and dried.

These warnings were based on research that showed that people who drank diet soda more often also tended to be more overweight, which may or may not be true. What if diet soda isn’t causing weight? But instead, obese people tend to switch to diet soda in hopes of losing weight. That is just one of a number of alternative hypotheses that could explain the correlation, and that’s why further more rigorous research is needed to identify the true cause.

In this case, that additional research has already been done, and scientists found that the correlation between obese people and drinking diet soda was in fact due to their efforts to lose weight. In other words, diet soda was more popular among overweight people trying to lose weight because it contains fewer calories than regular soda.

Furthermore, when it’s used in this fashion to reduce overall calorie, Diet soda consumption is associated with weight loss, not gain. Unfortunately, the media makes this type of mistake all the time. Studies show that news outlets tend to report on observational research more than randomized controlled trials, which can establish correlation and which you’ll learn more about soon, as well as lower quality studies that should be taken with a grain of salt, oversimplification, and sensational.

The media will often oversimplify or distort the results of a study. To make a nice catchy click bait headline, Tim Caulfield of the University of Alberta has coined a term for this science exploitation. For example, a popular UK website once ran the headline, A glass of red wine is the equivalent to an hour at the gym.

Says, New study with a sub-headline of new research reveals skipping the gym in favor of the pub Is. Perfect. Many people thought time to exercise less and drink more. If you actually read the scientific paper though, you’ll quickly realize that it isn’t what the study found. Instead, it found that a compound in grapes and red wine called resveratrol may increase exercise capacity.

How well people or animals tolerate intense. In rats who are already exercising. There was no red wine involved in this study and it never showed that people should stop working. Another example of this is when the media reported on a study from the New England Journal of Medicine with headlines claiming that drinking coffee could make you live longer.

However, not only did the media make the mistake of confusing correlation and causation, they also failed to mention that the study only involved people who had already lived to at least 50 years of age, had no history of cancer, heart disease, or stroke, and didn’t. There were many other limitations to the study as well, which the scientists mentioned in the paper, but the media failed to report on.

Why can the media get away with this? There are likely three reasons why this type of reporting continues Unabated. Journalists often have little formal training in science and thus are unable to ensure their stories are scientifically. The general public also has little formal training in science and is thus unable to differentiate good from bad reporting.

Sensationalism sells. So there’s always an incentive for the media to spend scientific research in sensationalistic ways. Keep in mind that most of these organizations rely on advertising revenue to survive, and advertising revenue is driven by website visits. Thus, from a business perspective, writing articles that get a lot of clicks is far more important than being scientifically accurate, especially when it would reduce the virality of the content.

Two cherry picking versus going by the weight of the evidence. It’s very common to have dozens or even hundreds of published studies on any given topic, and in many cases, the results aren’t all in agreement. Sometimes the differing or even contradictory results come from differences in how the studies were designed and executed.

Sometimes shenanigans are a foot and sometimes it’s just random. This is why scientists consider the weight of the evidence available as opposed to the findings of a single study. Think of a scale with one group of studies, more or less in agreement on one side and another group that indicates otherwise.

On the other side, the scale will favor whichever side has more evidence to support its assertion, which you could say is where the weight of the evidence lies. Thus, a good scientist will say, Given the weight of the evidence, this explanation is most likely. Unfortunately, due mainly to ignorance, personal biases, and the media’s love of controversy.

Research is often cherry picked to make claims that go against the weight of the evidence. In other words, people often pick out and play up studies that they don’t realize are flawed, that they just personally agree with, or that will make for great headlines. A perfect example of cherry picking occurs among some of the more zealous advocates of low carb dieting.

They often hold up a few studies as definitive proof that low carb diets are better for losing fat and claim. There’s no room left for discussion or debate. When you peruse these studies, though, you’ll find glaring flaws in how they were carried out. And when you collect and analyze all of the available research on the matter, you’ll find there is no practical difference in fat loss between low and high carb diets, so long as calories and protein intake are matched.

In other words, so long as people are eating the same amount of calories and protein, the amount of carbohydrate they’re eating won’t meaningfully impact their fat loss In the final analysis, dietary adherence, not carb intake is the biggest predictor of weight loss success. Thus a scientist worth their salt would say the weight of the evidence indicates that there are no differences in fat loss between low and high carb diets, so long as calories are restricted and protein intake is adequate.

Accordingly, individuals should choose the diet that they can best stick to for maximum results. Yep. The old adage is true. In many ways, the best weight loss diet is the one you can stick to three different quality levels of studies. As I mentioned earlier, there are often a large number of studies published on a particular topic, and some are better than others.

There are many factors to consider when assessing the quality of a study ranging from the type of research, observational or otherwise. It is to how well it’s designed, how many participants there were, whether humans or animals were involved, and more thus, when you’re working to determine the weight of the evidence, you have to consider not only the number of studies on each side, but the quality as well.

For example, if I have 10 studies with only 10 subjects, each that points to one conclusion as well as two studies with a thousand subjects each that points to another conclusion. Then the weight of the evidence lies with the latter, even though the former conclusion has more individual studies on its side.

As you’ll learn later in this course, sample size, which is the number of samples measured or observations used in a study is a major determinant of the quality of research. A perfect example of how ignoring the quality of research can result in misleading conclusions. Is antioxidant supplementation.

There’s low quality evidence in the form of observational research and small scale trials on animals and humans that suggests antioxidant supplementation may reduce the risk of cancer and high quality research in the form of randomized clinical trials that shows antioxidant supplementation. Guess which research the media and mainstream health gurus decided to champion?

Yep. The low quality research and antioxidant supplements started flying off the shelves. Four. Science moved slowly. Contradictions are a natural part of the scientific process. Many conclusions in science are tentative because they’re based on the best evidence available at the time. However, as time moves on and as scientists accumulate more data and evidence, newer findings and understandings can overturn older ones.

This is particularly true when there’s little data and evidence to begin. A good example of this process is the story of butter versus margarine. Three decades ago, as evidence accumulated that the saturated fat and butter may be related to heart disease, risk scientists recommended that people switch to margarine to reduce their saturated fat intake.

However, evidence then began to accumulate that the chemically modified fats trans fat in margarine were even worse than saturated fat in regard to heart disease risk. Based on this newer evidence, scientists revised their recommendations to continue to limit butter, but also eliminate margarine and trans fats from diets.

Five. Science often deals in shades of gray rather than black and. Science is full of nuance, and therefore research usually doesn’t lend itself well to headlines and sound bites, which is what most people want. Simple, neat, black or white answers to their questions. Unfortunately, though many scientific topics operate more in shades of gray, and especially when the evidence isn’t strong, there’s often a lot of uncertainty in the realm of science, which the general public finds uncomfortable.

They don’t want informed guesses. They want certainties that make their lives easier, and science is often unequipped to meet these. Moreover, the human body is fantastically complex and some scientific answers can never be provided in black or white terms. All this is why the media tends to oversimplify scientific research when presenting it to the public.

In their eyes, they’re just giving people what they want as opposed to offering more accurate, but complex information that very few people will read or underst. A perfect example of this is how people want definitive answers as to which foods are good and bad. Scientifically speaking, there are no good and bad foods.

Rather, food quality exists on a continuum, meaning that some foods are better than others when it comes to general health and wellbeing. Take sugar a molecule that most people consider. In and of itself, it’s not a harmful substance and one of its components is necessary for life. Glucose research shows that when it’s consumed in moderation as part of a calorie controlled diet, it doesn’t cause adverse health effects or fat game.

However, when sugar is added to highly processed foods to enhance their palatability and energy density, these foods become easier to overeat, and the resulting increase in calorie consumption and fat gain can become a health. That doesn’t make for a good tweet or elevator pitch to a book publisher though.

And so the research on sugar tends to be separated into two buckets, One that shows it’s good and another that shows it’s bad. This creates the illusion of incongruity when in fact it’s just a case of missing the forest for the trees.

If you like what I’m doing here on the podcast and elsewhere, definitely check out my sports nutrition company Legion, which thanks to the support of many people like you, is the leading brand of all natural sports supplements in the world. Six. Lack of reproducibility slash replica. A very important concept in the realm of science is replication or reproducibility.

For a scientific finding to be considered true, it needs to be reproduced, meaning that other scientists should be able to achieve the same results by repeating the experiment. This is important. Because if other scientists can’t replicate the results, then it’s likely the initial results were a fluke.

The media loves to report on hot new studies with new findings, but often such studies are small pilot experiments that have yet to be reproduced with larger sample sizes and better study designs. Often later studies end up refuting the results of the original breakthrough research giving the appearance of conflicting.

In reality, the initial results were happenstance. This is why it’s important to be cautious when viewing small studies with new or unusual finding. One of the greatest examples of this happened in the 1980s, two scientists held a press conference saying they’d been able to make Adam’s fuse at room temperature cold fusion.

However, they hadn’t reproduced their results and other scientists weren’t able to reproduce the results either. By late 1989, most scientists considered the prospect of cold fusion. It’s also important to take note of the labs conducting research. If one lab consistently produces a certain result, but other labs can’t reproduce it, then the research coming from the former lab should be viewed with skepticism.

For example, one lab has produced astounding muscle building results with a supplement hmb, but other labs haven’t been able to reproduce anything close, which calls the positive results into question seven. Poor Research design slash execut. Sometimes a study produces unusual results simply because it’s poorly designed and executed.

A perfect example of this is research out of the Ramini Institute in Italy that supposedly showed that Asan caused cancer in rats. The research was heavily criticized by leading scientific organizations for having many flaws, including the fact that the control rodents had unusually high cancer rates, and the fact that when independent scientists asked to double check the data, the institute flat out refused.

In most cases, organizations like this are outed among the scientific community, but by that time, the story has already made its way through the media cycle convincing many that once again, the scientific process doesn’t make any sense. Eight unpublished research. When scientists do a study, they collect the data, analyze it, write up the results, and submit the write up to a scientific journal for public.

The study then goes through a process of peer review, which consists of other independent scientists reviewing it for flaws based on their findings. The study is either accepted for publication or rejected. The peer review process isn’t without flaws, but it’s the first line of defense against bad research getting published and then propagated by the media.

Thanks to peer review. If a study is published in a scientific journal, you can at least know it’s gone through some type of quality. This isn’t the case with unpublished research. For example, scientists often present new research at conferences that has yet to be peer reviewed or published. Sometimes the media catches wind of these reports and runs with them before they’ve gone through the peer review process, and sometimes scientists will themselves promote the findings of studies that haven’t been peer reviewed or published.

One case of this was on September 30th, 2000. When Martin Lindstrom reported on his unpublished neuro imaging iPhone study in the New York Times, he reported that people experience the same feelings of love in response to their iPhones ringing as they did in the company of their partners best friends or parents.

Many scientists criticize Lindstrom stating that his data didn’t support such a. But since Lindstrom had bypassed peer review, his dubious conclusions were all that most people ever heard or saw. Companies that sell products often report unpublished research as authoritative proof of their effectiveness.

You should be wary of such research. Because it hasn’t been scrutinized by independent scientists and is often designed and executed in such a way as to guarantee positive results. For example, the creator of a cold exposure vest claimed that his product could help people burn up to 500 extra calories per day.

This startling promise was based on research he conducted himself, where people wore the vest for two weeks and lost fat. This trial was never peer reviewed or published in any scientific journal, and if it had been submitted for review, it would’ve been rejected for egregious design flaws. For instance, the alleged increase in energy expenditure was based on unreliable estimates of body composition rather than direct validated measurements of energy expenditure.

Nine. Fabricated research. Fabricated research essentially means research that’s been made up while fabricated research isn’t nearly as common as everything else we’ve covered so far. It still exists and can lead to great confusion. Scientists may falsify data for a number of reasons, including to gain money, notoriety, and funding for further research, or merely to add another publication to their.

One of the most famous cases of fabricated research came from Andrew Wakefield in 1988. He published a paper in the prestigious journal Lancet that showed an association between the measles mom’s rubella, MMR vaccine, and autism and children. However, it was later discovered that he had fabricated some of his data.

Independent researchers discovered that wakefield’s descriptions of the children’s medical cases differed from their actual medical records. Wakefield’s paper was eventually retracted from the journal, but to this day, his fraudulent research is still used to support the claim that vaccines may cause autism.

Despite numerous studies showing no such relationship, scientific research can seem like a quagmire of misinformation, contradiction, and outright lies. When you look under the hood though, you quickly find that the media selectively picks studies designed to generate the most con. Spins the findings for maximum dramatic effect and withholds information about how they were conducted.

In other cases, the shenanigans start before the studies hit your Facebook. Poor study designs skew the results and some scientists accidentally or intentionally falsify their data. Despite all of that, it’s still the best system we have for answering this simple question, what’s probably true and what isn’t.

To understand how honest, intelligent researchers go about answering that question. We need to take a closer look at the scientific. Key takeaways. The media often misrepresent scientific studies by confusing correlation with causation and oversimplifying and sensationalizing data. The media gets away with this because journalists and the public often have little formal training in science and sensationalism sells.

So there’s always an incentive for the media to spend scientific research. It’s very common to have dozens or even hundreds of published studies on any given topic, and in many cases, the results aren’t all in a. Unfortunately, due mainly to ignorance, personal biases, and the media’s love of controversy.

Research is often cherry picked to make claims that go against the weight of the evidence. In other words, people often pick out and play up studies that they don’t realize are flawed, that they just personally agree with, or that will make for great headlines. There are many factors to consider when assessing the quality of a study ranging from the type of research, observational or otherwise.

It is to how well it’s designed, how many participants there were, whether humans or animals were involved, and more thus when you’re working to determine the weight of the evidence. You have to consider not only the number of studies on each side, but the quality as well. Contradictions are a natural part of the scientific process.

Many conclusions in science are tentative because they’re based on the best evidence available at the time. However, as time moves on and as scientists accumulate more data and evidence, Newer findings and understandings can overturn older ones. This is particularly true when there’s little data and evidence to begin with.

For a scientific finding to be considered true, it needs to be reproduced, meaning that other scientists should be able to achieve the same results by repeating the experiment. This is important because if other scientists can’t replicate the results, then it’s likely the initial results were a. Peer review consists of other independent scientists reviewing research for flaws based on their findings.

The study is either accepted for publication or rejected. Scientists often present new research at conferences that has yet to be peer reviewed or published. Sometimes the media catches wind of these reports and runs with them before they’ve gone through the peer review process, and sometimes scientists will themselves promote the findings of studies that haven’t been peer reviewed or publish.

Fabricated research isn’t common, but it can lead to great confusion. Scientists may falsify data for a number of reasons, including to gain money, notoriety, and funding for further research, or merely to add another publication to their name. Despite its many flaws, the scientific process is still the best system we have for answering this simple question, what’s probably true and what isn.

All right. That’s it for this episode. I hope you enjoyed it and found it interesting and helpful. And if you did, and you don’t mind doing me a favor, please do leave a quick review on iTunes or. Wherever you’re listening to me from in whichever app you’re listening to me in, because that not only convinces people that they should check out the show, it also increases search visibility, and thus, it helps more people find their way to me and learn how to get fitter, leaner, stronger, healthier, and happier As.

And of course, if you want to be notified when the next episode goes live, then simply subscribe to the podcast and you won’t miss out on any new stuff. And if you didn’t like something about the show, please do shoot me an email at mike muscle for life.com. Just muscle f o r life.com and share your thoughts on how I can do this better.

I read everything myself, and I’m always looking for constructive feedback, even if it is c. I’m open to it and of course you can email me if you have positive feedback as well, or if you have questions really relating to anything that you think I could help you with, definitely send me an email. That is the best way to get ahold of me, Mike, at muscle life.com.

And that’s it. Thanks again for listening to this episode, and I hope to hear from you soon.