What an AI model actually does (and why it makes things up)

The one-sentence version

A useful way to picture a large language model: it writes by predicting what text should come next, one small piece at a time, based on patterns it learned from a very large amount of text and on whatever is in your conversation.

That is a simplification. Modern models are trained in several stages, can use tools, and have abilities no one-line description fully explains. But the picture is helpful for one important point: unless a tool looks something up for it, the model is generating a plausible answer rather than retrieving a stored fact. How often plausible is also correct varies a lot with the task and the model.

Where its knowledge comes from

A model learns during training, which happens before release. Each model has a training cutoff: a date after which it has learned nothing. Ask about anything newer and it can only guess, unless the tool you're using gives it fresh information through web search or files you add.

Providers publish these cutoff dates, and they differ between models from the same company. Our model profiles list the current ones with a link to where we read them.

The context window: its working memory

Everything the model can see while it answers is called its context window. That includes your messages, its earlier replies, any files you've added and the answer it is writing. Anthropic's documentation describes it as the model's working memory, separate from what it learned in training.

Windows are now large enough to hold several books, but more isn't automatically better. Anthropic notes that accuracy and recall can fall as the context fills up, an effect it calls context rot. Give the model what it needs for this task, not everything you have.

• Start a new chat when you change topic. Old material can pull the answer off course.
• Paste the relevant section of a long document, not the whole archive, when you can.
• If a long conversation starts going wrong, ask for a summary of what's agreed so far and continue from that in a fresh chat.

Why it makes things up

When a model states something false in a confident voice, it's usually called a hallucination. It's not a bug that will be patched next month. In September 2025, OpenAI published research arguing that common training and evaluation methods reward a model for guessing rather than saying it isn't sure, much like a multiple-choice test where a blank answer always scores zero.

Because the model is producing plausible text, its mistakes look exactly like its correct answers. The writing is just as fluent and just as confident. Some kinds of answer are much riskier than others:

Four habits that keep it honest

1. 1Give it the sourcePaste the text or upload the document and say "Answer only from this. If it isn't in the text, say so." Grounded answers are far easier to check.
2. 2Ask for quotesAsk it to quote the sentence each claim comes from. A quote you can find in the source is evidence. A quote you can't find is a warning.
3. 3Give it permission not to knowSay "If you're not sure, say so rather than guess." It won't always comply, but it changes what a good answer looks like.
4. 4Check what matters before it leaves your handsAnything going to a customer, a client, a publication or a decision gets checked against the original source by you.

Try it: answers with and without a source

What to remember

• Think of it as generating plausible text, not looking facts up, unless a tool fetches information for it.
• Its knowledge stops at a training cutoff. Recent or changing facts need a current source.
• Everything in the conversation counts toward its working memory. Keep it focused.
• Fluent and confident is not the same as correct. Check names, numbers, dates, quotes and citations.