Check an AI answer before you rely on it

Why fluent answers still need checking

AI tools write in the same confident voice whether they're right or wrong, so the usual signals you rely on, such as tone, detail and neat formatting, tell you nothing about accuracy. In a March 2025 study of eight AI search tools, Columbia Journalism Review's Tow Center found most of them "presented inaccurate answers with alarming confidence, rarely using qualifying phrases".

Checking doesn't mean distrusting everything. It means knowing which parts of an answer could hurt you if they're wrong, and checking those against something better than the AI's word. Our guide to what an AI model actually does explains why these mistakes happen; this one is about catching them.

The example we'll check

Here's the kind of paragraph an AI tool might give you if you asked whether AI search can be trusted. We wrote it ourselves for this guide and planted five mistakes in it. Try to spot them before you read on; the exercise at the end asks you to.

Step 1: Split it into claims

A paragraph hides mistakes because each sentence leans on the others. Pull it apart into single claims, one fact each. Facts, numbers, dates, names, quotes and citations need checking. Opinions and advice don't, though they're only as good as the facts under them.

Then sort by stakes. If you were about to repeat this in a report, the number, the quote and the citation are what a reader would hold you to, so they come first.

Step 2: Go to the original source

The original source is whoever produced the number or said the words, not an article about them. Figures get rounded, stretched and stripped of their conditions each time they're retold. Here, a search for the authors and "Tow Center" finds the study itself: "AI Search Has a Citation Problem" by Klaudia Jaźwińska and Aisvarya Chandrasekar, published by Columbia Journalism Review on 6 March 2025. Nieman Lab's write-up four days later is a useful lead, but the report is what you check against.

• Search for the specific number or phrase plus the organisation's name, not the AI's wording of it.
• Follow each article back until you reach the organisation that did the work: the study, the dataset, the official notice.
• Read the method, not just the headline. What was measured, on what, and when?

Step 3: Catch invented citations

AI tools can produce citations that look right and don't exist: a real author, a plausible title, a journal that sounds real. The same Tow Center study found that more than half of the responses from two of the tools "cited fabricated or broken URLs". Four quick checks catch most of them.

1. 1Search the exact titlePut the title in quotation marks in a search engine or Google Scholar, which recommends exactly this. Our example's title, 'Generative Search and the Death of Accuracy', finds nothing. That's claim 3 wrong.
2. 2Check the publication existsLook up the journal or site by name. There's no 'Journal of Digital Journalism' to be found. The study appeared in Columbia Journalism Review.
3. 3Open the link or DOIA DOI is a persistent identifier that should take you to the work itself. A link that leads to an error page, a homepage or a different article is a warning sign, not a typo to overlook.
4. 4Find the claim inside the sourceA real source can still be cited for something it doesn't say. Open it and find the sentence that supports the claim.

Step 4: Check numbers, quotes and dates

These are where small changes do the most damage. Check that the number measures the same thing, in the same group, at the same time, and that quotes are word for word.

Step 5: Spot a misleading summary

A summary can be accurate sentence by sentence and still mislead, because it drops the condition that changes the meaning. Claim 1, "AI search engines get it wrong most of the time", is the example's real problem. The study tested one task, finding the original news article for an excerpt, with each excerpt asked once, in tools as they were in February 2025. The authors say their findings "are not intended to be extrapolated to all models or news organizations."

Claim 5 checks out: the report lists eight tools including ChatGPT Search, Perplexity and Gemini. Five of the seven claims needed correcting, and the one that mattered most, the conclusion, was the hardest to see.

Checking an answer against your own documents

When the AI is summarising a document you have, such as a contract, a report or meeting notes, the check is easier because you hold the source.

• Ask it to quote the exact sentence behind each claim, then search the document for each quote.
• Ask what the document says about the limits, exceptions and dates, which are what summaries most often drop.
• Tools that answer only from documents you upload, and cite the passage, make this faster. You still need to click the citations.

Higher-risk answers, and when to ask an expert

Some answers deserve more than a search. Our judgment: if acting on a wrong answer could harm someone's health, money, legal position or safety, or if you're about to sign, submit or publish under your name, check it with a qualified person or the official body, not just another website or another AI.

• Health and medicines: a doctor or pharmacist.
• Legal rights, contracts and disputes: a solicitor, or an official advice service.
• Tax, benefits, pensions and investments: the official government source, or a regulated adviser.
• Anything safety-related, such as electrics, gas or structures: a qualified tradesperson.

Make it faster with the Claim Check

Steps 1 to 4 are the slow part. Our free Claim Check is one prompt that splits a piece of text into claims, gives each a verdict, and tells you which few to check yourself. It doesn't replace opening the sources: when your AI tool can't browse, it marks every verdict "FROM MEMORY - NOT CHECKED", which is your list of what still needs a source.

Try it

What to remember

• Checking reduces risk; it doesn't remove it. Sources can be wrong or out of date too.
• The study in our example tested tools as they were in February 2025, with each request asked once. Tools have changed since, so treat its figures as a warning about how confident errors look, not as today's error rate.
• Prioritise. You don't need to check every sentence, only the ones you'll repeat, rely on or be held to.