How to catch when AI is making things up about your health

AI hallucinations in health contexts are a specific problem because they inherit the authoritative tone of medical language. A language model can cite a study that doesn't exist, describe a biological mechanism that sounds plausible but is invented, or quote a statistic pulled from nowhere — all while sounding confident. Research published in JAMA Internal Medicine evaluated AI-generated medical answers and found that while often helpful, they sometimes contained fabricated citations presented with high confidence.

Hallucinations follow patterns. You can learn to spot them.

What hallucinations look like

Invented studies. AI generates something like: "Research from the Journal of Metabolic Health in 2022 found that magnesium supplementation reduces fatigue in 67% of cases." Specific journal, specific year, specific statistic. It has the shape of a real citation because AI learned the pattern from training data. But the journal may not exist, or the study may be fabricated.

The fix: search for it. Journal name plus year plus key terms in PubMed or Google Scholar. If you can't find it, it's likely fabricated.

Fabricated mechanisms. AI explains how something works in a way that sounds scientifically plausible but is actually invented. These are harder to catch. The pressure test: "Is this mechanism documented in medical literature?" If AI backtracks when pressed, the mechanism may be hallucinated.

False precision. "People with anxiety experience digestive issues 73% of the time." That level of precision implies a specific study. Real statistics in medicine are usually ranges or qualified descriptions. When AI gives a suspiciously precise number without a source, ask where it came from.

Overconfident rare diagnoses. AI confidently suggests an uncommon condition based on symptom matching. Rare conditions are rare — even if symptoms match, base rate probability makes common conditions far more likely until they've been ruled out.

How cross-checking helps (and its limits)

Iris uses a Supervisor that cross-checks responses across multiple AI models. When models disagree about a claim, the discrepancy gets flagged. However, if a hallucination is plausible enough that multiple models generate similar false information, the Supervisor may not catch it.

What to verify yourself

Citations: Search PubMed or Google Scholar. Real studies are findable.

Mechanisms: Ask whether the biological explanation is documented. Real mechanisms can be verified against standard medical references.

Statistics: If a number seems too precise or convenient, ask for the source.

Confidence signals: Watch for "It's well established that..." without evidence, or "doctors often miss this" (usually means it's not established consensus).

Hallucinations don't make AI useless for health investigation. They mean verification is part of the process — which is a healthy habit for any information source.

References

1. Evaluating AI-generated medical information — JAMA Internal Medicine, 2023. AI accuracy and hallucination patterns.
2. Large language models in medicine — Nature Medicine, 2023. Capabilities, limitations, and hallucination risks.
3. Sycophancy in AI language models — arXiv, 2023. AI tendency to agree with user framing.