Quick answer. AI hallucinations are confident, invented answers that sound plausible but are false, such as fake citations, invented package names or made-up facts. They happen because models predict likely-sounding text rather than check truth, and training rewards a confident guess over an honest "I don't know." You can reduce them by grounding answers in real sources, asking for citations and verifying anything you didn't check yourself.
What you should know about AI hallucinations
| Question | Answer |
|---|---|
| What it is | A confident, fabricated answer that sounds correct but isn't |
| Why it happens | Models predict plausible text and are trained to guess rather than abstain |
| Lowest summarization rate (2026) | 1.8%, Vectara leaderboard, May 2026 |
| Highest summarization rate (2026) | 24.2%, same leaderboard |
| Fake package names in code | 19.7% of recommended packages didn't exist, USENIX Security 2025 |
| Real-world cost | Lawyers sanctioned $1,000 to over $109,000 for fake AI citations in 2026 |
| Can it be eliminated | No, but it can be reduced with grounding, citations and verification |
| Best single fix | Give the model the real source document instead of asking from memory |
What is hallucination in generative AI?
Hallucination in generative AI is when a model produces an answer that is false or unsupported, but phrased with the same confidence as a correct one. The term applies across text, image and code models, and it covers a range of specific mistakes.
- Fabricated facts. A made-up date, statistic or quote attributed to a real person or source.
- Fake citations. A case, paper or study that doesn't exist, formatted to look real.
- Invented entities. A product, company, law or package name that was never real.
- Contradiction. An answer that conflicts with the source document it was supposed to summarize, or with an earlier part of its own reply.
- Overconfident extrapolation. A specific-sounding number or fact stated where the honest answer is "I don't know" or "this isn't in the source."
The word "hallucination" is a metaphor borrowed from psychology, and some researchers prefer "confabulation" because the model isn't perceiving anything, it's generating the statistically likely next words. Either way, when people say generative AI hallucinated, they mean it stated something false as if it were fact.
Why does AI hallucinate?
AI hallucinates because a language model is built to predict the next plausible word, not to check whether a statement is true, and its training rewards a confident guess over an honest refusal. A 2025 OpenAI paper, by researchers Adam Tauman Kalai, Ofir Nachum, Santosh Vempala and Edwin Zhang, traces this to two causes.
- Statistical pressure during pretraining. If a fact appears rarely or inconsistently in training data, for example a birthday mentioned once, the model can't reliably learn it. The paper notes that if 20% of some class of facts appear exactly once in training data, a model should be expected to hallucinate on at least that share of them, because there's no pattern to learn from a single mention.
- Evaluations that reward guessing. Most benchmarks grade answers as simply right or wrong, with no credit for saying "I don't know." The researchers call this a test-taking incentive: guessing sometimes gets the right answer, while abstaining never does, so a model trained to score well learns to guess rather than decline.
The paper's own example: when the authors asked a model for one of the paper's authors' birthdays, it confidently gave three different wrong dates rather than saying it didn't know.
This means hallucination isn't a bug limited to older or smaller models. It's a predictable result of how models are trained and graded, which is why the researchers argue the fix has to change the scoring of mainstream benchmarks, not just add a dedicated hallucination test.
How common are generative AI hallucinations?
How common hallucinations are depends heavily on the task, and the rate can range from under 2% to nearly 100% depending on what you measure. There's no single hallucination rate for AI, because different benchmarks test completely different things.
- Summarizing a document you give it. This is the easiest task. On Vectara's hallucination leaderboard, an industry benchmark on which a model summarizes a document and is checked for added facts the source doesn't support, rates across 108 to 109 tested models ranged from 1.8% to 24.2% as of May 2026, with a median around 9.6%. More than half the tested models stayed under 10%.
- Answering from memory, with no source given. This is much harder. Artificial Analysis's AA-Omniscience test measures how often a model gives a wrong answer instead of admitting uncertainty, and some models scored above 70% on this measure, meaning most of their wrong answers were confident guesses rather than an honest "I don't know."
- The same model, different tests. One comparison found GPT-6 Astra scored 4.2%, 8.7% and 51.3% depending on which of three different benchmarks was used. The lesson isn't which number is right, it's that the task you're testing matters more than the model's reputation.
The practical takeaway: a model is far less likely to hallucinate when it's working from a document you've given it than when it's answering purely from what it learned during training.
Real examples of AI hallucinations
The clearest documented examples are fake legal citations, which have cost lawyers real money and, in at least one case, a law license. Courts first ran into this in Mata v. Avianca in 2023, when a New York lawyer filed a brief citing six court cases that ChatGPT had invented, complete with fake quotes and docket numbers. A federal judge fined the lawyers $5,000.
The problem didn't go away. It escalated.
- Scale. A public database tracking these cases counted roughly 1,490 court decisions worldwide by May 2026 involving AI-hallucinated content, more than 1,000 of them in US courts.
- Bigger fines. In 2026, a Sixth Circuit panel sanctioned two attorneys $15,000 each, plus doubled costs and fees, for an appellate brief with more than two dozen fabricated citations. One Oregon attorney's repeated filings accumulated roughly $109,700 in sanctions across multiple cases.
- License loss. In September 2026, England's Solicitors Disciplinary Tribunal struck a foreign-registered lawyer off its register entirely, after he cited non-existent cases generated with AI. In April 2026, Nebraska's Supreme Court temporarily suspended an attorney after 57 of 63 citations in one brief turned out to be defective.
- Both sides can be wrong. In one Mississippi case in June 2026, a federal judge canceled a scheduled trial and sanctioned all four lawyers involved, because both sides had filed briefs containing AI-fabricated case law.
In all of these cases, the hallucinated text looked completely normal. Fabricated case names, realistic docket numbers and plausible legal language, all generated with the same confident tone as a real citation. Nothing about the output signaled it was invented, which is exactly why checking independently matters more than how confident an answer sounds.
AI code hallucinations
AI code hallucination is when a coding assistant invents something that doesn't exist, most often a software package name that sounds real but was never published. The landmark study on this, "We Have a Package for You!", presented at USENIX Security 2025, generated 2.23 million code samples across 16 models and found that 19.7% of recommended packages didn't exist, totaling 205,474 unique fabricated names.
This matters because of a specific attack called slopsquatting. Attackers watch which fake package names AI models invent, then register those exact names on real package registries like PyPI or npm. A developer who pastes AI-generated code and runs pip install or npm install without checking can unknowingly download and run an attacker's malicious package.
What makes this attack practical rather than theoretical:
- The hallucinations repeat. When researchers reran the same prompts 10 times, 43% of hallucinated package names came back on every single run, and 58% appeared more than once. A model's mistakes aren't random, they're predictable, which is exactly what an attacker needs.
- They look plausible. Only about 13% of hallucinated names were simple typos of real packages. Roughly 38% were "conflations," combining two real package names into a new, believable one, such as merging two legitimate libraries into a name that sounds like both.
- Checking isn't enough on its own. About 8.7% of hallucinated Python package names turned out to be real packages in the npm (JavaScript) registry, so a developer who checks "does this name exist anywhere" can still get a false sense of safety.
- Rates have dropped but not disappeared. A 2026 re-test of five newer frontier models found hallucination rates of 4.6% to 6.1%, down from the 2025 study's open-source average of 21.7%, but still producing 127 package names invented identically by all five models tested.
If you use AI to generate code with dependencies, verify every package name against the official registry before installing it, and don't assume a familiar-sounding name is safe.
How do you prevent AI hallucinations?
Prevention works best as a layered approach, because no single technique catches every hallucination. Builders and researchers currently use four main techniques, each tackling a different part of the problem.
| Technique | What it does | How effective |
|---|---|---|
| Retrieval-augmented generation (RAG) | Feeds the model real documents at the moment it answers, instead of relying only on what it learned during training | Strong for knowledge-intensive tasks; studies link it to reduced hallucination, though it depends on retrieval quality |
| Citation-enforced prompting | Requires the model to point to a specific source for every claim it makes | One medical-AI study found this cut hallucination rates to as low as 5.8% in a RAG pipeline |
| Chain-of-thought prompting | Asks the model to reason step by step before answering, rather than jumping to a conclusion | Reduces hallucinations in prompt-sensitive scenarios, though it doesn't fix deeper knowledge gaps |
| Confidence-aware training | Rewards a model for saying "I don't know" when uncertain, instead of only rewarding a lucky guess | The approach OpenAI's 2025 research recommends to fix benchmarks that currently reward guessing |
Retrieval-augmented generation (RAG) is the technique you'll see named most often, because it directly addresses the cause researchers point to: a model is far less likely to invent an answer when the real answer is sitting in front of it in the prompt.
AI hallucination prevention tips for everyday use
Most of what reduces hallucination risk doesn't require any technical setup, just a few habits when you're using a model directly.
- Give it the source instead of asking from memory. Paste the document, article or dataset into the chat and ask the model to work from that, rather than asking it to recall facts. This is the single biggest factor in the data above: summarizing a given document hallucinates far less than answering from memory.
- Ask for sources, then check them. If a model cites a study, case, or statistic, ask for the specific source and open it yourself. A citation that can't be found is a hallucination, however confident it sounded.
- Verify anything you'll act on. Treat names, numbers, package names, legal citations and quotes as unverified until you've checked them against a real source, especially before using them in anything public or official.
- Be specific and narrow. A vague, open-ended question gives the model more room to fill gaps with invented specifics. Ask for what the source actually says rather than for a general summary from memory.
- Compare two models on anything important. Different models hallucinate differently, and a second model, or a search of your own, can surface a disagreement that flags a problem.
- Treat fluent and confident as separate from correct. A hallucinated answer reads exactly like a correct one. Confidence in tone tells you nothing about whether a claim is true.
These habits don't eliminate hallucinations, since no current technique does. They lower how often you'll be the one who catches a fabricated fact after it's already caused a problem.
Can AI hallucinations be eliminated entirely?
No, not with current techniques. OpenAI's own 2025 research argues hallucinations aren't a fixable bug but a predictable result of how language models are trained and graded, since some facts in training data are too rare for any model to learn reliably. Even the best models on Vectara's easiest benchmark, grounded summarization, still hallucinate in 1.8% or more of summaries.
What changes is the rate, not the existence. Grounding a model in real sources, asking for citations and verifying claims independently all lower how often hallucinations occur and how much damage they do when they slip through. None of it gets the rate to zero, which is why verification stays necessary no matter which model or technique you use.
How can you check an AI answer across models on SYNTX.AI?
Send the same question to two or three different models and compare the answers, since models hallucinate differently and a disagreement is often the first sign something's wrong. On SYNTX.AI, GPT-6 Astra, Gemini 4 Argon, Claude Opus 5.5 and other text models share one token balance, so cross-checking a fact doesn't need a second subscription.
- Ask your question, and for anything important, attach the real source document instead of relying on the model's memory.
- Send the same prompt to a second model, such as switching from GPT-6 Astra to Gemini 4 Argon.
- If the answers agree and cite the same facts, you can trust it more. If they disagree, verify the claim independently before using it.
FAQ
What is hallucination when referring to generative AI?
It's when a generative AI model produces an answer that is false, fabricated or unsupported by its sources, but states it with the same confidence as a correct answer. It covers invented facts, fake citations and made-up entities across text, code and other outputs.
Why does AI hallucinate instead of saying it doesn't know?
Because training rewards a confident guess over an honest "I don't know." OpenAI's 2025 research found that most benchmarks grade answers as simply right or wrong with no credit for abstaining, so models learn that guessing scores better on average than admitting uncertainty.
How do you reduce AI hallucinations?
Give the model real source documents instead of asking it to answer from memory, ask it to cite specific sources, and verify anything you plan to use. Retrieval-augmented generation and citation-enforced prompting are the two techniques with the strongest evidence behind them.
What is AI code hallucination?
It's when a coding assistant invents something that doesn't exist, most often a software package name. A 2025 USENIX study found 19.7% of AI-recommended packages across 16 models didn't exist, which creates an attack called slopsquatting when someone registers the fake name with malicious code.
Can AI hallucinations be completely prevented?
No. Even the best-performing models on the easiest benchmarks still hallucinate in roughly 2% of cases or more. Grounding, citations and verification reduce how often it happens and how much harm it causes, but they don't eliminate it.



