How Large Language Models Actually Work

The surprisingly simple core idea

At the heart of every model like ChatGPT or Claude is one deceptively simple task: predict the next word. That's it. Given some text, the model's entire job is to guess what word comes next, then the next, then the next — one piece at a time — until it has produced a full response. Everything impressive these models do emerges from doing this one thing extraordinarily well.

Try it yourself. Finish this sentence: "The capital of France is ___." Your brain instantly filled in "Paris". A language model does the same, but for any text on almost any topic, having learned the patterns from a colossal amount of writing. When you ask it a question, it's really continuing the text in the most plausible way — and the most plausible continuation of a good question is a good answer.

Tokens: the pieces AI actually sees

Models don't quite work with words — they work with tokens, which are chunks of text. A token might be a whole word, part of a word, or a punctuation mark. "Hello" is one token; "unbelievable" might split into "un", "believ", "able". As a rough rule, one token is about four characters, and 100 tokens is roughly 75 words.

Why does this matter to you as a user? Two reasons. First, AI services charge by the token and have token limits — the "context window" is how many tokens the model can consider at once. Second, it explains quirks: models sometimes struggle with tasks like counting letters in a word, because they see tokens, not individual characters. Knowing AI thinks in tokens demystifies a lot of its odd behaviour.

Training: learning from the internet

How does a model learn which tokens follow which? Through training. Researchers feed it an enormous amount of text — books, articles, websites, code — and repeatedly play a fill-in-the-blank game. The model guesses the next token, gets told the real answer, and adjusts its internal numbers a tiny bit to be more accurate next time. Repeat this billions upon billions of times, and the model gradually internalises grammar, facts, reasoning patterns, coding conventions, and writing styles — all as statistical patterns.

The internal numbers it adjusts are called parameters. Modern large models have hundreds of billions of them. You can think of parameters as tiny knobs; training is the process of tuning all of them so the model's predictions match real human text as closely as possible. The "large" in Large Language Model refers exactly to this — an enormous number of parameters trained on an enormous amount of text.

From raw predictor to helpful assistant

A model trained only to predict text would just continue whatever you typed — not very useful. To turn it into a helpful assistant that answers rather than rambles, there's a second stage. Humans rate the model's responses, teaching it to be helpful, honest, and harmless. This stage, often called fine-tuning or alignment, is why ChatGPT responds like an assistant instead of just autocompleting your sentence. It's the difference between a raw engine and a finished, drivable car.

Understanding these two stages — broad training on text, then fine-tuning to be helpful — clears up a lot. It's why the model "knows" so much (it read a huge slice of the internet) but also why its knowledge has a cutoff date (it only knows what existed when it was trained) and why it can be confidently wrong (it learned patterns, not verified facts).

Why this explains everything else

Hold onto the "next-token predictor trained on text" model, because it explains the behaviour you'll see constantly:

• Hallucinations — the model generates plausible-sounding text, and sometimes the most plausible-sounding continuation is a made-up fact, citation, or name. It isn't lying; it's pattern-completing.
• Sensitivity to phrasing — because it continues your text, the words you choose steer it powerfully. This is why prompt engineering works.
• No real-time knowledge — it can't know today's news unless connected to a live tool, because its patterns are frozen at training time.
• Confidence regardless of correctness — fluency and accuracy are separate; it's always fluent, not always right.

None of this is a flaw to be feared — it's simply how the tool works. And now that you understand the mechanism, you're ready to learn the real skill: how to talk to these models to get exactly what you want. That's prompt engineering, and it's next.