An AI tool invents a source when its underlying process generates the kind of text that usually follows a claim, which is a citation, without any step that checks whether a matching real source exists. It is not lying, and it is not guessing in the way a person guesses. It is predicting the next plausible sequence of words, and a study name, an author, and a year are exactly the shape of words that tends to follow a factual-sounding claim in the text it was trained on. So it produces that shape, whether or not anything real sits behind it.
There is no "citation department" inside the model
It helps to be specific about what is not happening. The AI is not pulling a citation from a stored list of real sources and attaching it to your text. There is no retrieval step by default, no lookup, no cross-check against a database of actual publications, unless the tool has been specifically built with a separate retrieval system layered on top of the base model. Absent that, when you ask for a source, the model generates one the same way it generates every other sentence: by predicting what a plausible next chunk of text looks like, given everything that came before it.
A real academic citation has a recognizable shape: an author name, a year, a title, sometimes a journal or publisher. The model has seen millions of examples of that shape. It can reproduce the shape with total fluency, filling in details that sound right for the context, whether or not those specific details point to anything that exists.
Why it does this so confidently
Confidence and accuracy are two separate properties of the text a model produces, generated by the same process, and nothing in that process distinguishes them. A model is not weighing "how sure am I this is real" against "how sure am I this sounds right." It only has the second question. This is a known, structural property of how generative language models work, not a flaw specific to one product or one company, and it shows up across every major AI tool that generates open-ended text.
The three patterns this actually produces
- A citation invented from nothing. A study title, author, and year that do not correspond to any real publication, generated because that shape of text commonly follows the kind of claim being made.
- A real source, wrong claim. The cited study, article, or organization genuinely exists, but the specific finding attributed to it is not something that source actually reports. This is arguably the harder failure to catch, because the source checks out on a quick search.
- A real source, outdated framing. The citation is real and once said what the AI claims, but a newer version, a retraction, or an update has since changed the picture, and the model's training data predates that change.
All three look identical on the page. Only checking the actual source, and reading what it actually says, tells them apart.
Why asking the AI to "double-check itself" does not fix this
Requesting a second pass from the same tool does not add a verification step that was missing the first time. It generates another piece of plausible-sounding text, which may restate the same invented citation with equal confidence, invent a different one, or produce a caveat that sounds thorough without actually resolving anything. None of this replaces checking the source independently, outside the tool that produced the original claim.
What actually helps, and what it does not solve
Some AI tools are built with retrieval features that search real documents or the live web before answering, which measurably reduces invented citations compared to a model working from memory alone. This is a genuine improvement, not a gimmick, but it is not equivalent to human verification. A retrieval-augmented answer can still misread what a real source says, cite it out of context, or blend it with an unrelated invented detail. Treat retrieval features as lowering your workload, not eliminating your responsibility to check the specific claim yourself.
Why this is worse in some subjects than others
Invented citations cluster more heavily around topics where training data is thin, contested, or highly specific: niche industry statistics, very recent events, or narrowly scoped academic findings. Broad, heavily documented topics tend to produce fewer fully invented sources, simply because the model has seen far more real examples to draw patterns from. This is a useful signal for where to spend your verification effort, but it is not a safe filter for skipping verification altogether. A confident, specific-sounding citation on a broad, well-covered topic can still be entirely invented; it is just somewhat less common than on a niche one.
The practical takeaway
Every citation an AI tool produces starts in the same category: unverified. Not suspicious by default, not trustworthy by default, simply unverified until you look. That single habit of neutral suspicion, applied consistently rather than only when a claim feels "off," is what actually catches this. For the specific steps to run that check, see how to check whether a study an AI cited actually exists, and for the broader process this fits into, see how to fact-check anything an AI writes for you.
FAQ
Does this mean AI tools are being dishonest?
No, in the sense that dishonesty implies an intent to deceive, and there is no intent involved in how the text is generated. It is a structural consequence of predicting plausible text rather than retrieving verified facts. That does not make the invented citation any less false in your finished piece, but it changes how you should think about the failure and how to guard against it.
Are some topics more prone to invented citations than others?
Generally, the AI is more likely to invent detail in areas where its training data is thin, contradictory, or highly specific, such as niche statistics, recent events, or narrowly scoped studies. Broad, well-documented topics tend to produce fewer invented citations, though "fewer" is not "none," and treating any topic as immune is the mistake this whole discipline exists to prevent.
If a citation looks very specific, does that make it more trustworthy?
No, and this is a common trap. Specificity, such as an exact percentage or a precise-sounding study name, makes a claim feel more credible to a reader, but it says nothing about whether the model generated it from a real source or invented it wholesale with matching specificity.
Can I prevent this by asking the AI to only use real sources?
You can ask, and the instruction may somewhat reduce the rate of fully invented citations, but it does not give the model a new ability to verify existence that it did not previously have. The instruction changes the shape of what it generates, not whether a retrieval or verification step actually ran.
Is this the same issue as an AI misquoting someone?
It is a closely related version of the same underlying problem. A misquote is the model generating plausible-sounding words and attributing them to a real person, the same way it generates a plausible-sounding citation and attributes it to a real study. Both need the same fix: check the original, not the AI's restatement of it.