Today’s scientific studies are flawed. Here’s why.

If you are a regular consumer of morning television, you probably hear about scientific studies all of the time. If you watch NBC’s “Today” show, you may hear “Alcohol and Coffee May Help you Live Longer, Study Finds,” in which it is claimed “People who drank moderate amounts of alcohol or coffee lived longer than those who abstained.”  You might also discover an article titled, “More Than 5 Drinks a Week could Shorten Lives by Years, Study Finds,” in which it states,  “Any more than five drinks a week on average can take years off a person’s life.”

Why are these contradictions so common? Science is supposed to be factual and trustworthy. However, there are a lot of misleading scientific studies out there pretending to be legitimate, and it is leading to us putting too much trust into what we hear, which is harmful to society. The main problems with today’s experiments are the large numbers of them, errors made by scientists, lack of review, and the media’s distortion of facts.

The first problem is numbers. Results are being generated left and right, despite the amount of work needed to produce them. It takes Glenbard West science students weeks to write their lab reports, and, according to Mr. Kelly, West science teacher, “The process or practice of scientific studies in the science classroom at the high school level is to prepare students to understand the different aspects that go into scientific inquiry.” In the professional world, he explains, more work is required. That may suggest maybe not all of these results have gone through the full process of verification and are not rock solid. With so many studies in the public eye, there are a lot of contradictions. The one listed above is not the only example. The “American Journal of Clinical Nutrition,” a well-regarded and peer-reviewed journal, researched hundreds of nutritional studies and found that many foods are associated with both increases and decreases in cancer risk. For example, the “Journal” found four separate experiments which concluded that coffee protects against cancer, while another four separate experiments concluded that coffee increases the risk of contracting cancer. There are similar results for many other staple foods and drinks, such as milk and eggs. If you look at this data set, you might conclude that “all food causes and prevents cancer.” This conclusion provides no helpful information, and it can cast a cloud of uncertainty over all of these findings.

Many studies are performed poorly, as well, due to the fact that scientists are under a lot of pressure to produce new, flashy results. Dr. Brian Nosek from the Center for Open Science explained, “My success as a scientist depends on me publishing my findings, and I need to publish as frequently as possible in the most prestigious outlets that I can. Not everything gets published, so I’m incentivized to come up with positive results […] it has to be innovative.” This may sound troubling, but it makes a lot of sense. People are attracted to flashy conclusions, so scientists are incentivized to produce them. In order to avoid mediocre results, scientists can change their studies so they are more likely to produce a dramatic conclusion. Some changes can be small, such as shrinking the sample size or changing the length of an experiment. These subtle biases are not the only issue, however. “Not only subtle biases, but very strong biases can come into a study,” Mr. Kelly says. These strong biases include engaging in shadier activities, such as “P-hacking.” “P-hacking” involves scientists manipulating their data to create a statistically significant correlation by lowering the p-value of the results. This can turn meaningless data into strong evidence of a correlation. FiveThirtyEight, a website which focuses on surveys and polls, surveyed fifty-four people and collected over 1,000 variables, and used “P-hacking” to make statistically significant correlations that made absolutely no sense. For example, the website linked eating raw tomatoes to Judaism, and using table salt to having positive relationships with internet service providers. “Because of timelines, deadlines, and budget constraints, studies might not be completely conducted with the utmost care and reliability,” Mr. Kelly explains. If scientists do not have the ability to examine and eliminate their errors, the resulting studies can be lower in quality and harder to trust.

However, many of these flaws, regardless of whether they were made intentionally or not, go unnoticed. Replicating data, which can effectively confirm the results of an experiment, is very rare. This is troubling because, according to Mr. Kelly, a good study “has to be reproducible, and it has to have the ability for it to be reviewed, amended, adapted, and changed.” These replication studies are so rare, in part, because no one wants to do them. “Last Week Tonight” aired an interview with Dr. Elizabeth Iorns from Science Exchange, who stated that “Replication studies are so rarely funded, and they are so under appreciated, they never get published, no one wants to do them […] so you have all these exploratory studies out there that are taken as fact, that this is a scientific fact that’s never actually been confirmed.” Without the replication of findings, it is much easier for studies to be approved, even if they are not airtight.

However, this is not only the scientists’ fault. Many well-made studies presenting accurate information can become fallacious by the time they reach the public. The media is to blame for a large part of that. As mentioned before, scientists make more money when they produce flashy results. Media outlets are the same way. They can blow accurate data way out of proportion so that they are just as untrustworthy as the false data. An example comes from a peer-reviewed study about certain sources of hydrogen sulfite being “useful pharmacological tools to study the mitochondrial physiology of [hydrogen sulfite] in health and disease.” The study was supposed to be specific and only relate to finding the best ways to use hydrogen sulfite in health science. However, in July 2014, multiple news outlets, including “Time Magazine” and “The New York Post,” published articles about the study, with headlines such as “Time”’s “Scientists Say Smelling Farts Might Prevent Cancer,” even though neither flatulence nor cancer were mentioned anywhere in the study. The “Time” article has since been heavily corrected, but it is representative of a widespread problem with the media.

Now, many of you are probably wondering how it affects the public. You might think that as long as people view studies with some healthy skepticism it will be fine. I would argue that that is not entirely true. Mr. Kelly explained that “if the results of an experiment are misrepresented, [the public] can make decisions for their livelihood, for their health […] people’s choices can be swayed based on the results and how they are interpreted […] and that can be very dangerous in society.” Basically, there is no way to prevent everyone from not believing in fallacious data. People can make life-changing decisions based on fallacious science. In the late 1930s, Camel, a tobacco company, ran an advertisement stating, “Science confirms the fact that smoking Camels increases the flow of digestive fluids.” This was released decades before the public knew about the dangers of smoking, and it is likely some people decided to smoke because of that claim and others similar to it. 

Bad science is undoubtedly harmful to society, and we cannot stand idly by as it proliferates. So, before you place your trust in a scientific study, I implore you to do some research. Do not believe everything you hear and use basic common sense when analyzing scientific results. All of the problems with studies mentioned earlier, the contradictions, scientific errors, lack of review, and especially the media’s distortion of facts, can be minimized if we support the studies which are trustworthy. Paying attention to accurate findings will incentivize the scientific world to place more importance in the accuracy of its findings. Even just acknowledging this problem exists can make a difference. Hopefully, we can spread awareness of this problem, and move towards a better society.