Rigorous, high-quality scientific research is the basis for sound decision-making on the use, safety and development of chemicals.
Useful scientific evidence on the effects of chemical exposures can come from several types of experimental studies that are conducted in test tubes or petri dishes; within whole living organisms (e.g., rodents); or, increasingly, using computer models. Because such studies allow the investigators to precisely control the conditions of exposure, they can generate data that is considered reliable and easy to interpret.
Another type of evidence derives from observational studies conducted on people who have been exposed to chemicals at work, through unplanned events or in their everyday environments, also called “epidemiology studies.”
What is an Epidemiology Study?
Epidemiology studies look for patterns of disease and exposure in human populations, using data from employment records, questionnaires, health surveys, medical or death records and other sources. Epidemiology studies on workers and others who historically have had unusually high exposures to some substances (asbestos, for example) have been crucial to identifying relationships with certain diseases.
Factors Impacting Epidemiology Studies
Because epidemiology studies can only be conducted after people have been exposed to a chemical, they are not as useful as experimental studies for predicting and preventing adverse health effects:
- Epidemiology studies tend to produce less reliable data that can be more difficult to interpret. For instance, it is extremely rare that an epidemiology study alone can confirm that a particular chemical exposure caused a health effect. Moreover, as noted by Stanford University professor Dr. John Ioannidis, the vast majority of published epidemiology studies are later proven to be wrong.
- Furthermore, with the exception of randomized-control trials that research potentially beneficial therapeutic pharmaceuticals, it is considered unethical to randomly allocate humans into exposed and unexposed groups. So epidemiologists can only observe, not control, the conditions under which people are exposed. Consequently, a vast, unknown number of other variables, referred to as confounders, may cloud our understanding of the relationship between a chemical exposure and observed health effects.
- Errors in measurements of exposure and disease also can occur, which can further skew findings. Potential confounding and measurement errors are especially problematic in studies that include a relatively small number of subjects (i.e., less than tens of thousands), report exposures near background levels and report weaker relationships (e.g., increased risks less than 3 fold between groups). Finally, not all epidemiology study designs produce equally strong evidence (see Figure, “Evidence Hierarchy of Epidemiological Study Design”).
How to Evaluate Epidemiology Studies
Despite these limitations, given their potential value, researchers will continue to conduct epidemiology studies. The public needs to understand how to evaluate and interpret these studies so they can make informed decisions about their health.
Here are a few tips to help you evaluate and interpret such studies and their findings:
- Ready every story and study with a heavy dose of skepticism! Don’t just take them at face value. Useful guides to help you evaluate studies include 20 Top Tips for Interpreting Scientific Claims and the Rough Guide to Spotting Bad Science.
- Try to locate and read the actual scientific study that served as the underlying basis for a media story. Don’t be intimidated as a layperson — you may be surprised — a lot of it is just common sense. A useful step-by-step guide for non-scientists describes how to interpret study results.
- As you are reading the study, ask yourself:
- What question(s) is the study trying to answer?
- What study design was used, and how much does it deviate from a controlled experiment?
- Were sufficient numbers of people included in the study?
- Were potential confounding variables adequately controlled?
- How was exposure to chemicals measured and how accurate is it likely to be?
- How was disease measured and how accurate is it likely to be?
- How strong are the reported relationships between exposure and disease? Increased risks less than 3- or 4-fold are considered weak to moderate by experts and often have alternative explanations.
- How much exposure would it take to cause the alleged health effect? Is it realistic to receive such a dose in the normal course of events?
- What is my overall impression of the study?
- Independent replication by other scientists is the hallmark of good science. Consistent findings reported by independent investigators — preferably using different methods — should be accorded greater weight.
