LLM City
How chatbots think — a beginner’s tour
Large language models aren’t magic. They’re cities of tiny bets.
Chatbots don’t “think” like people. They repeatedly guess the next little piece of text. Do that thousands of times and it can look like understanding.
Six stops · plain English · step by step
First, chop the sentence into bricks
You read “spaceship” as one idea. A language model doesn’t start there. It turns your text into tokens — small chunks it can count and compare.
- Why? Computers are great with lists of numbers. Tokens are how text becomes that list.
- Not always whole words. Long or rare words often get split into pieces.
- Try it: edit the sentence below and watch the bricks change.
Tip: try a long word like “internationalization” — you’ll often see it split.
Each colored brick is one token. Inside the model, each brick is really just an ID number — the neural net never “sees” your sentence as a human would.
The model’s only job: guess the next brick
After it reads the tokens so far, it doesn’t jump to a finished paragraph. It asks one question over and over: “What token is most likely next?”
- Not a search engine. It’s not looking up a saved answer — it’s making a bet.
- Tallest bar = favorite. But it can still pick a shorter bar sometimes.
- Try it: press Roll the next token and watch the story grow one brick at a time.
The tallest bar is the “safe” choice. Picking other bars is how writing stays interesting — and sometimes wrong.
Press Roll to pick one token from these odds. That’s called sampling.
Which earlier words should we care about?
Before guessing the next token, the model looks backward. Attention is how it decides which past words matter most right now.
- Pink outline = “I’m predicting the token after this word.”
- Brighter blue = more attention (more useful for this guess).
- Try it: click different words and watch the spotlight jump.
Click any word to move the “cursor.” The bars on the right show who gets the spotlight.
Spotlight budget
Attention is like a fixed budget of “focus points.” The bars below show how that budget is split across earlier words.
Real models run many spotlights at once (attention heads) — one might track names, another grammar, another quotes. Same idea, lots of copies.
The context window is a backpack, not a brain
Everything the model can use for this answer must fit in a limited list of tokens: instructions, chat history, and what it’s writing now. That list is the context window.
How “spicy” should the next guess be?
Remember those probability bars? Temperature reshapes them before the model picks. Same favorites, different willingness to take risks.
- Low (≈0.2): almost always the top choice — good for facts & code.
- Medium (≈0.8): a mix — common for chat.
- High (≈1.5): wilder words show up more — fun, less reliable.
Drag the slider, then sample. At low temp, “mat” should win almost every time.
If the chips all say the same word, temperature is low. If they’re mixed, temperature is high.
Put it together: chat like a model
This is a tiny toy chatbot for learning — not ChatGPT. Real models are huge and much smarter, but they still follow the same basic loop.
Lower = safer, similar replies. Higher = more random wording. Same idea as District 5.
What just happened?
After each reply, read this board top to bottom. It’s the whole tour in one glance.
1 · Tokens (bricks)
Send a message to fill this in.
2 · Attention (spotlight)
Which words got more focus?
3 · Next-word bets
Which first words were most likely?
Click a suggested message, then match each board section to a district you already visited.