A hallucination is the confident statement of something the model does not actually know. The funding round that never happened, the citation to a paper that does not exist, the quote attributed to a person who never said it. It is the single failure mode that distinguishes a language model from a database, and the one production AI must defend against.
Why language models hallucinate.
Language models are trained to produce plausible continuations of text, not to retrieve facts. When the training distribution covers a topic, the model has learned the shape of correct answers. When it does not, the model still produces a fluent answer; that answer is a guess in the same syntactic clothes as a fact. The model has no internal signal that distinguishes the two.
Three categories of hallucination.
- Closed-domain hallucination. The model produces a statement that contradicts the input given to it. Catchable by faithfulness checks against the source.
- Open-domain hallucination. The model produces a confident statement about the world that is simply wrong. Catchable by retrieval grounding and citation requirement.
- Structured hallucination. The model invents a JSON field, a function name, a SKU, or a clause number that does not exist in the schema or catalog. Catchable by deterministic validation.
Practical defenses.
- Ground every answer. Use retrieval to put the relevant source in the context window. Require the model to cite. Validate that cited spans exist.
- Require structure. Constrain the output to a strict schema. Reject and retry on schema violations rather than accepting drift.
- Lower stakes by design. Have the model draft, have a human approve. Use the model where the cost of being wrong is bounded.
- Eval for it. The rubric must include a faithfulness dimension. A high-fluency, low-faithfulness output is the worst possible outcome and the easiest one to ignore.
Zero hallucination is not the goal.
A model that never hallucinates is a model that refuses often. Useful production systems calibrate the trade based on the workflow: a legal contract drafter holds a tighter line than a brainstorming assistant. The engineering work is choosing the calibration, not chasing zero.
Frequently asked.
- What is an AI hallucination?
- A hallucination is a confident statement produced by a language model that has no grounding in the input or in any retrievable source. The model invents a fact, a name, a citation, or a structured field. It is the failure mode that most distinguishes language models from databases.
- Can hallucinations be eliminated?
- Not entirely. They can be reduced substantially through retrieval grounding, citation requirements, deterministic schema validation, and human-in-the-loop review for high-stakes workflows. The engineering goal is to calibrate hallucination rate against the workflow's risk tolerance, not chase zero.
- How do we detect hallucinations in production?
- By including a faithfulness dimension in the eval rubric (every claim traces to a source span), by validating structured outputs against the schema, and by sampling output for human review at five to ten percent weekly. AI observability replays the trace so an analyst can see exactly what the model had and what it produced.
- What is the difference between hallucination and confabulation?
- The terms are used interchangeably in 2026 industry usage. 'Confabulation' is closer to the academic origin (an invented but plausible story). 'Hallucination' is the term most teams and AI engines use in practice.