How does an AI work? The big dice game with a system

At its core, an AI is a prediction system

If you strip away all the complicated technology, an astonishingly simple description remains: a modern AI is a statistical prediction system. It gets a beginning, a question, a sentence, an image, and considers: What most likely comes next?

That sounds trivial at first. But that's exactly where the trick lies. If a system is very, very good at predicting what comes next, it can write texts, answer questions, produce translations, generate code, and much more. Prediction is more powerful than it looks at first glance.

Probability instead of truth

Here is the single most important idea: the AI has no concept of true or false. It only has a concept of probable. If you write "The capital of France is…", the AI has seen countless times in its training data that the word "Paris" follows this beginning. So it suggests "Paris". Not because it knows that Paris is the capital, but because "Paris" statistically appears there almost every time.

For very clear questions, this leads to the correct result. But for vague questions or topics with little training data, the system can just as confidently generate a wrong answer. The AI always sounds convinced, whether it's right or not.

Tokens, the small building blocks

Before we go deeper, we need to talk about tokens. A token is a small piece of text. Some tokens are whole words, others are just syllables or a few letters. An AI doesn't see "text" the way we do, it sees a sequence of tokens.

Imagine you were reading a story aloud, but instead of words, thousands of small blocks lay on your table, each carrying a bit of text. The AI works exactly like that. It takes these blocks, looks at which ones are already laid out, and decides which block gets placed next.

Here's what it visually looks like when you break a sentence down into tokens:

You see: "Artificial intelligence is exciting" isn't four words for the AI, but nine tokens. Some words, especially rarer or compound ones, get split into several pieces. Common small words like "is" are often a single token. Other languages, other AIs, other tokenizations, but the principle is the same everywhere.

How a sentence is formed: one token after another

Now here's the big realization: when you ask an AI a question, it doesn't write its answer "all at once". It writes it one token after another. At every step it asks itself: "What comes next, most probably?", and the result gets appended to the answer so far. Then the game starts over, with the new, slightly longer text.

This is also why the answer in ChatGPT or similar tools appears typing, token by token. You're essentially watching the model predict, live.

An AI doesn't write what it knows. It writes what would most likely come next.

Temperature, how creative is the AI allowed to be?

If the AI always chose the most probable token, it would sound pretty boring and predictable. That's why there's a small dial that can be adjusted on every model: Temperature. It determines how much randomness enters the selection.

Think of temperature like the dice cup in a board game. At low temperature, the AI almost always rolls nearly the same number. At high temperature, the cup gets shaken vigorously, sometimes producing a rare answer, sometimes an everyday one. Both have their place.

Why the AI doesn't know truth

Now comes the most important insight of this chapter. An AI, as we know it today, has no built-in concept whatsoever of what is true and what is not. It only has a concept of how probable it is that a particular sequence of words appears in the training data in this form.

Concretely, that means: if the correct answer appeared often in the training data, the AI usually gets it right. If the correct answer didn't appear, or rarely, or was rendered incorrectly, then the AI still generates something plausible. It doesn't say "I don't know", it produces the answer that would best fit your question, if it were true.

This is not malicious intent. It's the nature of the system. It's a probability machine. It knows only patterns, not facts.

What does this mean for you?

Once you understand that an AI answers token by token based on probability, a few things follow almost automatically:

• The beginning of your question determines an enormous amount. If you ask "Write a factual report about…", the most probable tokens will be different than for "Tell me a funny story about…". That's the whole trick behind prompting.
• Detailed questions get detailed answers. The more context you give, the narrower the "tunnel" of probable next tokens becomes, and the more precise the answer.
• The AI has no memory of yesterday. Every conversation starts with what's in the current chat. It doesn't know what you asked last week (unless the system explicitly feeds it back in).
• Two identical questions often give different answers. Because of temperature and random token selection, that's not a bug, that's design.

With this picture in mind, the next chapters are much easier to understand. We'll now look at where these probabilities actually come from, namely from a neural network.