Understanding science

Whether a nutrition claim is true can rarely be answered with a single study. This page explains the terms and ways of thinking we use in all fact checks.

How studies work

Nutrition research uses very different types of studies. They answer different questions and are prone to different errors.

• Randomised controlled trial (RCT): participants are assigned to groups by chance, for example “more eggs” and “no eggs”. Because of randomisation, the groups differ on average only in what is being tested. RCTs are therefore best at showing cause and effect. In nutrition, however, they are often short and small, and they usually measure blood values rather than diseases.
• Cohort study: many people are followed for years. They are asked what they eat, and illnesses are counted later. Such studies are large and long, but people chose their diet themselves. People who eat a lot of a particular food often differ in many other ways too.
• Case-control study: people with a disease are compared with healthy people and asked about their past diet. This is prone to recall errors.
• Cross-sectional study: a snapshot. It shows associations, but not what came first.
• Measurement or residue study: food samples are analysed for substances. It shows what was found in particular samples at a particular time and place, but not automatically whether people become ill from it.
• Systematic review and meta-analysis: all suitable studies on a question are searched for and assessed according to fixed rules and, where possible, combined statistically.
• Animal and cell studies: they explain possible mechanisms but do not allow direct conclusions about human health.

Why correlation does not prove causation

When two things occur together, it does not mean that one causes the other. A well-known example: on days when a lot of ice cream is sold, more people drown. Ice cream does not cause drowning. Both depend on warm weather.

In nutrition studies, such background factors are called confounders. People who drink a lot of soft drinks, for example, may also smoke more often or exercise less. Good studies adjust for such factors, but never completely. What remains is called residual confounding.

A second problem is reverse causation: people who are already ill often change their diet. It can then look as if the new diet caused the illness.

An association becomes more credible if it is strong, appears in many independent studies, increases with the amount consumed (dose–response relationship), fits in time, is biologically plausible and is confirmed in experiments. None of these points alone is proof.

What meta-analyses can and cannot do

A meta-analysis combines the results of many studies statistically. This makes the result more precise and shows whether studies point in the same direction.

However, a meta-analysis is only as good as the studies it contains. If it pools many biased observational studies, the overall result is biased too, just with less uncertainty. Heterogeneity also matters: if individual studies differ greatly, a single average value means little.

Several meta-analyses on the same topic often contain the same studies. Three meta-analyses are therefore not automatically three independent confirmations. We check for this overlap and do not count it more than once.

What confidence intervals mean

Studies often report results like this: “hazard ratio 1.06 (95% confidence interval 1.03–1.10)”. The first value is the best estimate: a 6% higher relative risk. The interval in brackets shows the range in which the true value very probably lies, provided the study has no systematic errors.

• If the value 1.0 (“no difference”) lies within the interval, the finding is not statistically established.
• A narrow interval means a precise estimate; a wide interval means an uncertain one.
• Statistically significant does not automatically mean important. A very small effect can be “significant” in a huge study and still hardly matter in everyday life.
• The interval only captures chance, not confounding or measurement error.

Why single studies can mislead

Any single study can reach a result that is not confirmed later, because of chance, small samples, many questions tested at once, or a special population. On top of this comes publication bias: striking results are more likely to be published than unremarkable ones.

We therefore do not assess whether some study supports a claim. With a selective choice, almost any claim can be “proven” with studies. What matters is the totality of good evidence.

Absolute and relative risks

A relative risk says by what percentage a risk rises or falls compared with another group. An absolute risk says how many people are actually affected.

Worked example (invented numbers for illustration): Of 1,000 people, 2 develop a certain disease within ten years. In another group it is 3 out of 1,000. The relative risk is 50% higher. But the absolute risk rises by only 1 in 1,000 people, i.e. by 0.1 percentage points.

Both statements are correct but feel very different. Where available, we therefore give both. If a study does not report absolute numbers, we say so.

Detection is not the same as danger

Modern laboratories can detect tiny amounts of substances, often in the range of billionths of a gram. The fact that a substance is found says nothing yet about whether it is harmful in that amount. What matters is:

• The amount actually taken in (exposure).
• Health-based guidance values, such as the tolerable weekly intake (TWI) or the acceptable daily intake (ADI). Authorities such as the European Food Safety Authority (EFSA) set them with safety margins.
• Maximum residue levels (MRLs): legal upper limits. Exceeding one is a legal violation but does not automatically mean a health hazard. Conversely, a value below the limit is not meaningless for every person and every combination of substances.
• The context: a measurement from a contamination incident, a mining area or another country does not automatically describe the normal food supply in Germany or Egypt.

The reverse is also true: for some substances, such as certain per- and polyfluoroalkyl substances (PFAS), EFSA has set very low guidance values (2020: 4.4 nanograms per kilogram of body weight per week for the sum of four PFAS). Here, a normal diet can bring some people into the range of the guidance value. That is why we examine each case individually.

Animal and cell studies

Experiments on cells or animals show whether a mechanism is possible. However, doses there are often much higher than in the diet, and animals process substances differently from humans. We always label such studies explicitly. Biological plausibility is not clinical proof.

How our assessments are made

Each claim receives two separate ratings:

1. Verdict on a seven-point scale from “True” to “False”, including “Unclear / insufficient evidence”. A lack of evidence does not mean that a claim is false.
2. Certainty of evidence (high, moderate, low, very low), based on the internationally used GRADE system. It tells how confident we can be about the overall evidence. Observational studies start lower than randomised trials and are downgraded further for shortcomings.

We break broad claims down into sub-claims. “Eggs harm everyone” can be refuted, while “certain people should limit their egg intake” can be correct. We also rate individual studies along several dimensions, such as methodological quality, risk of bias, precision and regional transferability, and we check whether they were reported correctly.

We use tools such as RoB 2, ROBINS-E, AMSTAR 2 and GRADE for orientation. A complete formal assessment with these tools requires full texts, study protocols and usually several independent reviewers. That was not possible here. Our judgements are therefore explicitly labelled as simplified methodological assessments.

Evidence status, evidence dossiers and depth of checking

Since the systematic evidence review (phase 4), each claim also has an evidence status. It states in words what kind of answer science gives:

• Scientifically refuted: high-quality evidence shows the opposite, or the claim contradicts well-established physiology.
• Not supported by high-quality evidence: there is no good evidence for the claim. This alone does not prove that it is wrong.
• Overinterpreted: a real finding exists, but the claim draws more from it than the data allow – for example an association presented as a cause, animal data applied to humans, or a small effect presented as a large one.
• True only under certain conditions: the claim holds only for certain amounts, groups of people, preparations or circumstances.
• Insufficiently studied: there is too little research to say whether the claim is right or wrong.
• Scientifically well supported: the claim agrees with consistent, high-quality evidence.

When high-quality evidence clearly refutes a sweeping medical claim, we say so plainly: “This claim is false.” Where data are contradictory or insufficient, we do not claim more certainty than the evidence allows. A correct statement is rated as correct, even if other statements of the same system are wrong.

For every claim there is an evidence dossier: a concrete research question (structured as PICO for interventions or PECO for exposures: population, intervention or exposure, comparison, outcomes), documented searches in PubMed, Europe PMC, the Cochrane Library and other sources – separately for supporting and for contradicting evidence –, the strongest studies on both sides, an explanation of apparent contradictions, a statement on whether regional differences matter, and the reasons for keeping or changing the verdict. Where studies publish enough data, we recalculate key results (for example absolute risks from event counts); otherwise we say that a recalculation was not possible. Missing data are never filled in.

We do not count publications. Several meta-analyses that contain the same primary studies are not independent confirmations. PRISMA 2020 is a standard for transparent reporting of systematic reviews, not a tool for rating study quality. GRADE rates the certainty of the body of evidence for one outcome and does not replace the assessment of individual studies (RoB 2 for randomised trials, ROBINS-I and ROBINS-E for non-randomised studies, AMSTAR 2 and ROBIS for systematic reviews).

For every study we state how deeply it was checked: bibliographically verified (the study exists; title, authors, year and identifier match), abstract checked, full text checked, or full text with a structured risk-of-bias assessment. No study has yet been formally assessed by two independent reviewers, and no independent human expert review has taken place. We never describe a check as formal or human if it did not happen.