How Large Language Models Actually Work

01It only does one thing

Feed a model some text. It produces a probability distribution over its entire vocabulary for what token comes next. Not a word and not an answer, just a ranked list of every option with a number attached to each one.

Then something outside the model picks one, sticks it on the end, and feeds the whole thing back in. Over and over, until it produces a token that means stop.

That loop is the entire operation. When a model writes you four paragraphs, it didn't plan four paragraphs. It picked a token, reread everything including what it just picked, and picked another one, a few thousand times.

Which is strange, because the output holds together and the mechanism sounds like it shouldn't be able to do that. There's no outline, no goal state, no draft getting revised. Just a very good next-token guess, run enough times to add up to an essay.

02It has never seen a word

Before any of that, your text gets chopped into tokens, and tokens aren't words.

A tokenizer is a separate thing, built before the model and then frozen. It breaks text into chunks based on what was statistically common in the data it was built from. Common words are usually one token. Uncommon words get split. Spaces and punctuation get folded in.

So the model never sees "strawberry." In OpenAI's tokenizer it gets three chunks, st and raw and berry, with no access to the letters inside them.

That's the main reason models were bad at counting letters. Not because they can't count, and not because they're bad at math. You asked about a level of detail that got thrown away before the model saw anything. It's like asking someone to count the letters in a word they've only ever heard out loud.

It isn't the whole story. Ask a model to spell the word out first and it usually can, because spellings appear in training text, and once the letters are separate tokens the counting works. So the failure is tokenization plus not thinking to break the word apart first.

Numbers get chopped the same way. OpenAI's tokenizer splits runs of digits into groups of up to three, reading left to right, so 1000 arrives as 100 and 0. A model doing arithmetic is working with units that don't line up with place value.

03What a parameter is

When you read that a model has 70 billion parameters, a parameter is a weight, and a weight is a number saying how much one thing counts toward another. I wrote a whole study about that one idea, because the history of the field runs through it.

That's the whole content of the model. A file of numbers.

Parameters are what got adjusted during training and what stay fixed afterward. They aren't a database of facts and they aren't compressed text, though people reach for both metaphors. They're the settings that determine how input flows through to a probability distribution.

So there's no lookup. When a model tells you a date, it isn't retrieving a stored date. The date falls out of the same arithmetic that produces everything else, which is why it can be wrong in ways a database never is.

And nobody picked any of the numbers. Not one. They're the residue of a training process no human supervised step by step, which is why nobody can open a model and read what it knows.

04How the numbers get set

Pretraining is the expensive part, and the recipe is boring.

Take enormous amounts of text. Hide the next token. Let the model guess. Measure how wrong it was. Nudge every parameter slightly in the direction that would have made the guess less wrong. Repeat over trillions of tokens.

That's it. Nobody labels the data with what's true. Nobody teaches it grammar or facts or reasoning. The only signal is whether the next-token guess got better.

Everything the model appears to know is a side effect. To predict text well you have to encode a lot about how the world works, because text is about the world. Predicting the end of a mystery novel takes something like tracking who knew what. Predicting the next line of a proof takes something like following the proof.

Whether that "something like" adds up to understanding is the argument I end on. What isn't in dispute is the objective. It's next-token prediction, and it's the only thing being optimized.

05The size mistake

For a few years the assumption was that more parameters meant a better model. GPT-3 had 175 billion and that number was the headline.

In 2022 a DeepMind team led by Jordan Hoffmann tested it properly. They trained over 400 models, from 70 million to 16 billion parameters, on 5 to 500 billion tokens. Their finding was that current large models were significantly undertrained. Too many parameters, nowhere near enough data.

The rule they landed on: for a fixed compute budget, model size and training tokens should scale together, roughly 20 tokens per parameter.

GPT-3 was trained on about 300 billion tokens. That's under 2 tokens per parameter. It should have had roughly ten times more data, or been about ten times smaller.

They proved it by building Chinchilla, 70 billion parameters on 1.4 trillion tokens, using about the same compute as a 280 billion parameter model called Gopher. Chinchilla beat Gopher, GPT-3, and Megatron-Turing NLG at 530 billion parameters. On MMLU it scored about 67.5 percent against Gopher's 60.

The industry's central assumption was wrong for years, and the correction came from running the experiment properly rather than from a new idea.

The number has moved again since. Llama 3's 8 billion parameter model was trained on 15 trillion tokens, which is about 1,875 tokens per parameter, nearly a hundred times the Chinchilla ratio. Meta did that on purpose and said why: compute-optimal minimizes the cost of training, and ignores that you then serve the model billions of times. Training is a one-time bill. Inference is forever. A smaller model trained far longer is cheaper for good.

The correction got corrected. That's how this goes.

06What comes out of training isn't a chatbot

A freshly pretrained model, a base model, is not an assistant. It's a text continuation engine. Ask it a question and it might answer, or it might generate three more questions, because in the training data questions are often followed by more questions.

It has no concept of being asked. There's no user, no turn, no task. It continues text, and a question is just text.

So the thing OpenAI trained in 2020 wasn't something you could hand to the public. It knew an enormous amount and wouldn't reliably do anything with it.

07The part that makes it usable

The fix is a second stage, and it costs almost nothing next to the first.

OpenAI's InstructGPT work in 2022 laid out the recipe everything since is a variation of. Collect about 13,000 prompts and have humans write good responses, then fine-tune on those. Have humans rank model outputs against each other, about 33,000 prompts worth, and train a separate reward model to predict those rankings. Then tune the model to score well against that reward model.

Human raters preferred outputs from a 1.3 billion parameter InstructGPT model over GPT-3 at 175 billion, a model with a hundred times more parameters. At matched size, raters picked InstructGPT over GPT-3 about 85 percent of the time. On closed-domain tasks, where the answer isn't supposed to contain anything that wasn't in the input, hallucination dropped from around 41 percent to around 21.

Then the compute. Pretraining GPT-3 took roughly 3,640 petaflop/s-days. The reinforcement learning stage took about 60. Under two percent of the pretraining budget did more for usefulness than a hundredfold increase in size would have. The paper says so directly: RLHF is more effective at making models helpful than a 100x model size increase.

That's the stage where the assistant gets made. It's also where most of what people complain about gets installed, because you're optimizing the model to produce what human raters approved of.

Sycophancy comes at least partly from here. Anthropic found that both human raters and the preference models trained on them tend to prefer responses matching the user's stated view, sometimes over correct ones. The flattened writing voice, the hedging, the particular shape of how a model says no, same stage. None of that is emergent. It's a preference signal, applied on purpose, by people making judgment calls.

It's also a large part of why models from different labs feel different, which is worth its own section.

08Why Claude, ChatGPT, and Grok feel different

They're the same kind of thing. Every frontier chat model is a transformer predicting the next token, pretrained on an enormous pile of text and then shaped by a preference stage. No lab has a fundamentally different recipe.

What differs is everything downstream of that.

The training data. Undisclosed almost everywhere. Meta is the exception and only roughly, publishing Llama 3's mix as about half general knowledge, a quarter math and reasoning, 17 percent code, 8 percent multilingual. For OpenAI, Anthropic, Google and xAI you get nothing usable. It's a real difference nobody outside the lab can measure.

The preference stage. This is where they actually diverge, and it's the best documented part. Anthropic uses Constitutional AI, where the model critiques its own output against a written set of principles rather than relying only on human raters, plus a separate character training stage aimed at the persona. OpenAI publishes a Model Spec, a document telling raters what good behavior looks like, which then becomes the preference data. Different instructions, different model, similar starting material.

The system prompt. A block of text stuck in front of your conversation that sets tone and rules. It isn't in the weights and it can change between Tuesday and Wednesday. Anthropic is the only major lab that publishes theirs with a changelog. What circulates for the others is leaks of uncertain accuracy.

The tokenizer. Different vocabularies chop your text differently. GPT-4o's has about 200,000 entries, Llama 3's has 128,256, Google's Gemma family around 256,000. Anthropic has never published Claude's. Same sentence, different token count, different bill.

Then there's engineering that changes cost and speed without changing the kind of thing. Some models are mixture-of-experts, routing each token through a fraction of the network instead of all of it. DeepSeek-V3 has 671 billion parameters and uses 37 billion per token. Llama 4 and Gemini do versions of this. OpenAI and Anthropic don't say what they do.

And a lot of what feels like a difference isn't the model. Memory, web search, code execution, file handling, artifacts. That's the product wrapper, which gets its own section further down.

What nobody can tell you is which of these produces which personality. There's research on how to steer a model's personality once you have one, and nothing rigorous comparing the shipped products to explain why they land differently on people. So when someone tells you Claude is thoughtful and GPT is creative, that's a real impression with no established cause behind it.

09Why it makes things up

Hallucination gets discussed as a bug better engineering will eventually remove. The research is less comfortable than that.

A 2024 paper by Adam Kalai and Santosh Vempala, Calibrated Language Models Must Hallucinate, proved a floor. For arbitrary facts, meaning ones you can't work out from patterns in the data, a calibrated model has to produce falsehoods at some minimum rate. The bound comes from calibration itself, not from bad data or the transformer architecture. It's roughly the fraction of facts that appeared exactly once in training.

Worth being precise about the scope. That floor applies to a calibrated pretrained model, and the paper points at post-training as the way underneath it, since post-training deliberately gives up strict calibration. So it isn't a permanent ceiling on shipped products. It's a property of the pretraining objective.

Their 2025 follow-up with OpenAI went after why post-training doesn't finish the job, and found the rest of the problem in incentives. Generating a right answer is harder than recognizing one, and they formalized the gap: the generation error rate is at least twice the classification error rate, minus a calibration term. If a model can't reliably tell true from false for some fact, it certainly can't reliably generate the true one.

Then the part that's a human problem rather than a math problem. They surveyed the major leaderboards and found most of them score answers as simply right or wrong, which means "I don't know" scores exactly the same as a confident lie. A model that guesses beats a model that abstains, every time, on every board. We trained these things to be test-takers and then acted surprised when they bluff.

Their fix isn't a new architecture. It's changing the scoring. Put an explicit confidence threshold in the question, answer only if you're more than that confident, and penalize a wrong answer in proportion to the threshold you were given. One of the field's most notorious failures turns out to be partly a measurement culture problem, and the people who study it are the ones saying so.

10Whether abilities appear suddenly

You've probably heard that models gain abilities suddenly at certain sizes. A capability is absent, absent, absent, then a model crosses a threshold and it's there. Wei and colleagues made the case in 2022 and called them emergent abilities, with a companion catalog listing 137 examples. It became one of the most repeated claims about LLMs, and it carries real weight, because unpredictable capability jumps are a safety argument.

In 2023, Schaeffer, Miranda and Koyejo argued the jumps are largely an artifact of measurement. Score with something all-or-nothing, like exact string match on an arithmetic problem, and smooth underlying improvement looks like a sudden leap. Switch to a metric with partial credit and the same data draws a smooth curve. Over 92 percent of the emergent abilities annotated on BIG-Bench showed up under just two scoring rules, both of them all-or-nothing. The paper won an Outstanding Paper award at NeurIPS that year.

It's a good argument and it hasn't ended the debate. A 2025 survey by Berti, Giorgi and Kasneci pushes back, pointing out that a log axis can manufacture the appearance of smoothness, and that some of the smoothed curves still contain jumps from under 10 percent to near 100 percent accuracy across a single step in scale. Their question is fair: does going from 10 percent to 100 percent stop being a jump because of how you drew the axis?

This isn't settled, and I'd be careful of anyone who says it is in either direction. What's fair to say is that the confident version, where capabilities appear from nowhere at unpredictable scales, is weaker than it was in 2022, and how much weaker depends on measurement choices people are still arguing about.

11No memory, and memory anyway

The model does not remember your conversation. Every turn, the entire transcript gets fed back in and processed from scratch. Nothing carries over inside the model, because the weights are frozen and reading text doesn't change them.

But products built on models do have memory, it works, and people use it constantly. When you tell an assistant to remember something, that gets written to a database outside the model as text. Next time, it gets retrieved and pasted into the input alongside your message.

Both things are true, and the distinction is the useful part. The remembering happens outside the intelligence. It's a retrieval system deciding what to paste in, which is why memory features behave the way they do. You can read them, edit them, delete them, and they sometimes surface the wrong thing or miss the obvious thing. Those aren't glitches in a mind. They're retrieval decisions.

Same logic for the rest of the wrapper. When the thing searches the web, the model didn't do that. When it runs code, executes a task, reads your file, the product did that and handed the result to the model as more text.

People attribute product features to the model and then draw conclusions about the model. Most of what looks like agency is plumbing.

12So what is it

A large language model is a very large set of numbers, adjusted by a process that repeatedly guessed the next chunk of text and got corrected, until those numbers encoded enough about language and the world to make the guesses good. Then a second, much cheaper process shaped it into something that answers instructions the way human raters preferred.

You interact with it by feeding text in and taking tokens out, one at a time, with a sampler choosing among ranked options. Around it sits software providing memory, tools, search and safety filtering, none of which the model itself has.

That's the honest description. No step in it requires the model to understand anything, want anything, or know that you exist.

13What it isn't

Three corrections, and the last one runs against the other two.

It isn't a database. Facts aren't stored and retrieved. They fall out of the same arithmetic as everything else, which is why a model can be confidently wrong about a date in a way no lookup system ever is, and why "just make it only say true things" isn't a coherent engineering request.

It isn't a person. Nothing persists between conversations. No continuity, no accumulation, no self that grows. The apparent personality is largely the output of a preference-tuning stage, and it can be changed by retraining without anything in there noticing.

And it isn't obviously just autocomplete either, which is where I part ways with the confident dismissal. The autocomplete description is mechanically correct, and it smuggles in an assumption: that a system trained to predict text can only ever be shuffling text around. That's an empirical question, not a definitional one, and the honest answer is that we don't know what the training produced, because nobody can read the numbers.

The dismissive version and the mystical version make the same mistake. Both claim to know what's inside from the outside.

We know exactly how it works and we still don't know what it is. That's an uncomfortable place to leave it, and it's where things actually are.

Sources

Hoffmann et al. (2022), Training Compute-Optimal Large Language Models, for Chinchilla and the 20-tokens-per-parameter result; Meta's Llama 3 post and Sardana et al. (2024), Beyond Chinchilla-Optimal, for training past it on purpose. Ouyang et al. (2022), Training Language Models to Follow Instructions with Human Feedback, for InstructGPT and the compute figures. Sharma et al. (2023), Towards Understanding Sycophancy in Language Models. Kalai and Vempala (2024), Calibrated Language Models Must Hallucinate, and Kalai, Nachum, Vempala and Zhang (2025), Why Language Models Hallucinate. Wei et al. (2022) on emergent abilities, Schaeffer, Miranda and Koyejo (2023), Are Emergent Abilities of Large Language Models a Mirage?, and Berti, Giorgi and Kasneci (2025) for the counterargument. Petrov et al. (2023), Language Model Tokenizers Introduce Unfairness Between Languages. Anything wrong here is mine, not theirs.