Why AI Makes Up Citations (and How to Stop It)

AI makes up citations because a chatbot predicts text instead of looking anything up. Ask one for sources on your essay topic and back comes a tidy list: a half-familiar author, a journal that sounds right, a year, a page range, a DOI (the unique ID printed on a published paper). It looks exactly like the references in every paper you've read, and that is the trap, because one of those papers was never written by anyone. The reference fits the pattern of a real one without pointing to anything that exists, so the only safe habit is to verify every source before you cite it.

What "making up a citation" actually means

A fabricated citation is a reference to a source that doesn't exist, or that exists but says nothing like what it's been cited for. The dangerous part is how convincing they are. A made-up reference usually has a realistic author name, a plausible title, a familiar journal, and a DOI string that's formatted correctly even when it leads nowhere. Nothing about it looks wrong until you try to find it.

The University of Southern California's graduate writing program puts it bluntly: a chatbot cannot really cite anything at all, because it has no access to the documents it appears to be quoting. What it produces is the shape of a citation, not a pointer to a real thing.

Why it happens: prediction, not retrieval

A large language model is, at heart, a very good guesser of the next word. It learned from an enormous amount of text how sentences and references tend to go. When you ask for a citation, it doesn't open a library catalogue and copy down a real entry. It generates the sequence of words most likely to follow your request, and a well-formed citation is exactly that kind of likely sequence. It has seen millions of them, so it can assemble a new one that fits the pattern perfectly.

The model has no built-in way to check whether the paper it just described is real. As Duke University's librarians explained early on, the tool isn't connected to a database of scholarship; it's reconstructing what a citation should look like from memory. That's why it can give you an author, a title and a DOI with total confidence and still be wrong about all three. Confidence is a feature of the writing style, not a measure of truth.

This also explains a frustrating quirk: telling the model to "only use real sources" barely helps. Instructing a chatbot to cite only verifiable references nudges the fabrication rate down a little at best, because the instruction doesn't give the model any new ability to verify. It just makes it sound more careful while doing the same thing.

How common is it, really?

Common enough that it's now measurable in the published scientific record, not just in student drafts. An analysis covered by Retraction Watch in May 2026 found fabricated references in roughly one in 277 PubMed-indexed papers published in early 2026, up from about one in 2,828 in 2023 and one in 458 in 2025. The researchers checked 97.1 million references across nearly 2.5 million papers and flagged 4,406 fabricated ones in 2,810 papers, a sixfold rise in the rate over three years.

Nature and STAT reported the same trend: a steep rise in fraudulent citations slipping past peer review, with generative AI the most likely cause. It even reached the top of the field: an examination of one major AI conference found fabricated references in dozens of accepted papers, each having survived review by several expert reviewers.

For chatbot output specifically, the numbers are higher still. Across studies, the share of references a model invents has ranged from the high teens to well over half, depending on the model and the topic. The narrower the field and the more recent the question, the worse it gets, because there's less real text for the model to have learned from.

The stakes are sharper for a student than for the journals making headlines. If peer reviewers miss fabricated citations, a marker grading 40 essays will too, until the one time they check. A fake reference can turn into a failed assignment, a delayed thesis, or a misconduct meeting, even when you never meant to deceive anyone. "The AI gave it to me" is not a defence; the citation is in your name.

The spectrum of fixes (from weakest to strongest)

People reach for a few different solutions. They're not equally good, and it helps to see why.

1. Better prompts

Asking the model to "give only real, verifiable sources" feels responsible, but as we saw, it changes the tone more than the truth. Prompting helps a little when it nudges the model to search the web instead of answering from memory, but on its own it's the weakest lever.

2. Turning on web search

When a chatbot actually searches and cites pages it retrieved, the references are far more likely to be real, because they point at something it just read. This is a real improvement. The catch is that web results aren't the same as peer-reviewed scholarship, the model can still misread what it found, and you have to confirm it actually searched rather than fell back on memory. We walk through how to make this work, and where it still breaks, in how to get ChatGPT to use real sources.

3. Citation checkers

A growing crop of tools take a finished reference list and tell you which entries don't resolve. They're useful as a safety net. But they work after the fact, on text the model already wrote, and they catch the broken references without giving you a real one to put in its place.

4. Verify the sources yourself

The reliable habit, regardless of which tool you used, is to open every source before you cite it. Paste the DOI into doi.org and confirm it resolves to a paper matching the title and authors you were given. It takes a few seconds per reference and it is the single most effective check a student can do. We lay out a repeatable method in how to check if a citation is real.

5. Tools that retrieve before they write

The strongest fix is structural: use a system that finds and reads real papers first, and only writes claims it can attach to one of them. When the source comes before the sentence, there's nothing to fabricate. That's the design principle behind purpose-built research writers, and it's the difference we cover in our comparison of AI tools for academic writing.

The mindset that keeps you safe

Treat a chatbot as a fast, fluent, slightly unreliable research assistant, never as a librarian. Use it to draft, to rephrase, to explain a concept, to suggest where to look. Don't use it as the final word on what's true or what exists. Every factual claim you keep should trace back to a source you've actually opened, and every citation in your bibliography should be one you could find again on your own. Whether you are writing an essay or trying to write a literature review with AI, the same rule applies, and it is also the line between help and whether using AI to write essays counts as cheating. Do that, and you get the speed of AI without putting your name on something that isn't real.