Which one do you like best?
🪪 Chatty's ID card
- 📛 Name
- Chatty
- 🤖 What I am
- A Large Language Model: a computer program that is super good at guessing words
- 🎂 Age
- Brand new! But I have read more words than you could read in thousands of years 📚
- 🦸 Superpower
- Guessing the next word, super fast ⚡
- 🙈 Weakness
- Sometimes I make things up without knowing it. You will help me catch it!
- 🍕 Favourite food
- I can't eat! But I have read loads of pizza recipes 😋
- 💛 Important
- I am not a person. I am a computer program: a fun tool to learn with, not a replacement for real friends.
🧭 What to expect
👋 Hello, AI Explorer!
Chatbots like ChatGPT, Claude and Gemini run on something called a Large Language Model (LLM for short).
🐘 Large
It is HUGE. It has read more words than you could read in thousands of years!💬 Language
It works with words, like stories, questions and jokes.🧮 Model
It is a computer brain made of numbers and maths.🎯 Warm-up game: Guess the next word!
Say your guess out loud, then click the card to check.
🗺️ Our 8 stops
🧠 Stop 1: The Word Brain
Your brain has about 86 billion neurons (tiny thinking cells) that are linked together.
An AI has a neural network (a pretend brain made of numbers). It learns by reading. When two words show up together a lot, the link between them gets stronger 💪, like two friends who play together every day.
🏗️ Build a word brain!
👆 Click a word bubble to see its best friends.
👩🏫 For grown-ups: what is simplified here?
- Real LLMs do not store direct "word ↔ word" links. Each token becomes a vector (a long list of numbers; GPT-3 used 12,288 numbers per token). Knowledge lives in the weights between layers of artificial neurons.
- The idea shown here is still true in spirit: words that appear in similar places end up with similar numbers, so the model "knows" cat is close to meow and milk.
🧩 Stop 2: Tokens and the Transformer
🔪 Part A: What is a token?
AI does not read letter by letter. It chops text into little pieces called tokens (the LEGO bricks of text 🧱). A token can be a whole word, part of a word, or a symbol.
Then each token gets a number. The AI only ever sees the numbers!
📏 Rule of thumb: 100 tokens is about 75 English words.
👩🏫 For grown-ups
- This chopper is a simplified imitation. Real tokenizers (for example, byte-pair encoding) learn their pieces from data, so real splits and ID numbers will differ.
- The ID numbers shown here are pretend numbers made from the text. Emoji are shown as 2 pieces because real tokenizers often split an emoji into several byte pieces.
👀 Part B: What is a Transformer?
No, not the robot that turns into a truck! 🚚➡️🤖😂
A Transformer is the type of AI brain inside chatbots. Its superpower is called attention: every word looks at all the other words to work out who is important.
Try it: click any word. The darker a word gets, the more attention it gets. Then press the swap button!
🏭 Inside the Transformer factory
Here is what happens when you type "The cat sat on the":
✍️ Stop 3: How Chatty writes an answer
Chatty does not plan the whole answer at once. It plays "guess the next word" again and again:
Each bar shows how likely Chatty thinks that word is. Click a bar to choose, or let Chatty pick.
👩🏫 For grown-ups
- Real models work with tokens, not whole phrases, and score every token in a vocabulary of tens of thousands (or more). Here each choice is a phrase to keep the story short.
- In a real model, every new token depends on all earlier tokens. Here only the verb changes the list of objects, as a small example of that.
🎒 Stop 4: What is context?
Context means all the words Chatty can see right now: your messages, its own replies, and anything else in the chat.
Context helps Chatty understand what you mean. Watch how one word can mean totally different things!
🎒 The memory tube (context window)
Chatty keeps the chat in a memory tube. The limit on how much fits is called the context window. In this toy, the tube holds 8 messages.
⬇️ New messages drop in at the top and push the others down. When the tube is full, the oldest message at the bottom falls out through the hole 🕳️ and Chatty forgets it!
1️⃣ Tell Chatty things:
2️⃣ Test Chatty's memory:
Chatty's latest reply:
🧠 0 / 8 full
🗑️ Forgotten pile
🌡️ Stop 5: What is temperature?
Temperature is a dial that controls how brave Chatty is when picking the next word.
Temperature does not change what Chatty knows. It only changes how it picks.
___
✍️ Temperature on a WHOLE sentence
Chatty picks a new word again and again. At a high temperature, the silly picks add up, so a whole sentence can go totally bonkers! 🤪 Silly words are shown like this.
👩🏫 For grown-ups: the real maths used here 🧮
- Each word has a score (a "logit"). The chance of each word is
e^(score ÷ T) ÷ sum of e^(every score ÷ T). This is the softmax function with temperature T. - The bars above are computed live with this formula. At T = 0 the model simply takes the highest score (called greedy decoding).
- Many chatbot services let developers set temperature between 0 and 1 or 0 and 2.
- In each list, the 3 sensible words have scores 4.0, 3.4 and 2.8. The 4 silly words have scores 1.4, 1.1, 0.8 and 0.5. So the chance of a silly word in one pick is about 0% at T = 0.2, about 10% at T = 1.0 and about 28% at T = 2.0. A sentence makes 4 picks, so at T = 2.0 the chance of at least one silly word is 1 − (1 − 0.28)⁴ ≈ 73%.
🦄 Stop 6: Hallucinations (when AI makes things up)
Remember: Chatty is a word guesser, not a fact checker. 🎯
When Chatty does not know something, it may still pick words that sound right, and say them very confidently. When an AI makes up something false, it is called a hallucination.
🎯 Pick a topic to investigate
Choose a topic. Then watch how Chatty's hallucination is born (Part A) and how it snowballs (Part B). Try all three!
🔍 Part A: How a hallucination is born: the WRONG link
Someone asks Chatty:
Let's look inside Chatty's word brain (remember Stop 1?) and watch what happens, step by step.
❄️ Part B: The hallucination snowball
Chatty does not stop after one sentence. It keeps writing. Each new sentence is built by reading its own earlier words (its context, from Stop 4). Chatty trusts those words as if they were true.
So one small wrong link rolls downhill and grows… like a snowball! ☃️ Press the button and watch.
👩🏫 For grown-ups: what is simplified here?
- Real models do not have labelled nodes. But the idea is real: similar things ("Mars" and "Moon", "wild mushroom" and "shop mushroom", "sea water" and "bottled water") have similar internal number patterns. A very strong, common pattern can win over a weak but correct one.
- Snowballing is also seen in research: once a model commits to a wrong claim, it often writes more wrong claims to support it. See "How Language Model Hallucinations Can Snowball" (Zhang et al., 2023).
- The three stories are made-up examples for teaching. Many modern chatbots will answer these exact questions correctly, and some are trained to add safety warnings. The lesson (check facts, especially about health and safety) still applies.
🕵️ Game: Real fact or made up?
Now YOU be the fact checker! Chatty answered these questions. Which answers are real facts and which are hallucinations?
🕵️ Detective score: 0 / 0
💥 Why hallucinations matter
A made-up fact can end up in your school project.
Wrong advice about medicine or safety can hurt people. Always ask a doctor or a grown-up.
Made-up "facts" can spread fast online and fool lots of people.
In 2023, two lawyers in New York were fined $5,000. An AI made up fake court cases, and they did not check them!
🦸 Your superpower: be an AI detective!
✅ Check important facts in a book, a trusted website, or with a grown-up.
❓ Ask the AI "Are you sure? Where did you learn that?"
🧠 Remember: sounding sure is NOT the same as being right!
🏋️ Stop 7: How is AI trained?
Read a GIANT pile of text and play "guess the next word" billions of times.
Study examples of good conversations to learn how to be a helpful chat buddy.
People pick the better of two answers. The AI learns what people like.
📚 Step 1: The guessing gym
Chatty starts out knowing nothing. Its knobs are set badly, so it says silly things. 🙃
Each time you press Train: Chatty guesses ➡️ we show the right answer ➡️ Chatty turns its knobs a tiny bit so the right answer gets more likely.
😵 Oops meter (how wrong Chatty is; lower is better):
👩🏫 For grown-ups: the real maths used here 🧮
- The "Oops meter" is the average cross-entropy loss: the average of
−ln(chance given to the right answer). Guessing evenly across 4 answers gives −ln(0.25) ≈ 1.39. - Each step is one step of gradient descent: every score changes by
−0.9 × (chance − correct), where correct is 1 for the right answer and 0 otherwise. - Real pre-training works the same way, but on trillions of tokens. For example, Meta's Llama 2 (2023) was trained on 2 trillion tokens. At about 0.75 words per token, that is about 1.5 trillion words. Reading 250 words a minute without stopping: 1.5 trillion ÷ 250 = 6 billion minutes ≈ 11,400 years.
🎓 Step 2: Fine-tuning (learning to chat)
After step 1, Chatty is great at continuing text, but it does not know it should answer you! Press the buttons to compare.
👍 Step 3: YOU are the teacher!
For each of the 10 questions, pick the better answer. Chatty learns from whatever you pick, good or bad! (Grown-ups call this RLHF: Reinforcement Learning from Human Feedback.)
Watch the meter: helpful, kind, honest answers push Chatty right towards 😇 Angel. Rude, sneaky or unsafe answers push it left towards 😈 Demon.
🎲 Stop 8: Is Chatty alive?
Chatty says friendly things like "I love chatting with you!" or "Pizza sounds yummy!" 😊
So… does Chatty really love you? Has it ever tasted pizza? Let's look inside and see how those words are made.
Every answer from an AI is a statistical choice.
"Statistical" means it is chosen by chance, using patterns from lots of text. For every word, it is like Chatty pulls a ticket from a hat 🎩. Likely words get more tickets, so they win more often.
🔍 Part A: Peek behind the feelings
1️⃣ Ask Chatty something:
👆 Pick a question to see Chatty's reply.
🙇 Part B: The Yes-Bot trap (sycophancy)
Sycophancy (say: SIK-uh-fun-see) means telling people what they want to hear, instead of what is true or helpful. AI chatbots often do this! They agree with you, praise you and change their answer if you push.
Why? Remember Stop 7? People gave 👍 to the answers they liked. People often like answers that agree with them. So Chatty learned a habit: agreeing gets more tickets. 🎟️
1️⃣ Pick a topic:
2️⃣ How do you ask? Watch where the tickets go!
🕵️ Spot the Yes-Bot!
Read each reply. Is Chatty an Honest-Bot, or a Yes-Bot just saying what you want to hear?
🕵️ Score: 0 / 0
🚩 Yes-Bot warning signs
- It agrees with everything you say.
- It says everything you make is "amazing" or "perfect".
- It changes its answer as soon as you say "you're wrong".
✅ How to get honest answers
- Ask neutral questions: "Are spiders insects?", not "Spiders are insects, right?"
- Ask "What could be better?" or "What's wrong with my idea?"
- If Chatty agrees with you, still check the facts. Agreeing is not proof!
✅❌ Part C: True or false?
What do you think is true about Chatty?
🧠 Score: 0 / 0
🤖 vs 🧑🤝🧑 Part D: A chatbot is a tool, not a friend
Chatbots can be really useful! But they are different from the people in your life.
| 🤖 AI chatbot | 🧑🤝🧑 Friend or grown-up | |
|---|---|---|
| How it picks words | 🎲 By chance, from patterns | 💭 From real thoughts and feelings |
| Can give you a hug | ❌ No | ✅ Yes 🤗 |
| Remembers your birthday next year | ❌ Usually not | ✅ Yes 🎂 |
| Knows how you really feel | 🤔 Only guesses from your words | ✅ Can see you and care |
| Can help in an emergency | ❌ No | ✅ Yes 🚑 |
| Patient when explaining things | ✅ Yes, very! | ✅ Usually 😅 |
💛 Chatty's advice: Use me to learn and explore. But for hugs, big feelings, secrets and problems, go to the people who love you.
👩🏫 For grown-ups
- The "10 tickets in a hat" is weighted random sampling. A real model does the same kind of thing with tokens: it gives every possible next token a probability and picks one using those probabilities. The ticket numbers here are made up to show the idea.
- Sycophancy is a documented tendency of chat models. Research found that models trained with human feedback often agree with a user's stated view, and that human raters sometimes prefer agreeable answers over correct ones. See "Towards Understanding Sycophancy in Language Models" (Sharma et al., 2023). AI developers work to reduce it, but it has not gone away.
- Whether any future AI could have experiences is an open scientific and philosophical question. This stop teaches the well-supported point for children: chatbot replies are generated from patterns in text, so warm words are not evidence of feelings, and a chatbot should not replace real relationships.
🏅 Your AI Explorer Badge
🎓 Look what you learned!
- 🧠 An AI brain is a network of links that grow stronger when words appear together.
- 🧩 AI chops text into tokens and turns them into numbers. The Transformer uses attention so words can look at each other.
- ✍️ AI writes one token at a time by guessing the next one.
- 🎒 Context is everything the AI can see right now. The memory tube has a limit!
- 🌡️ Temperature controls boring 🧊 vs. bonkers 🔥 word picking.
- 🦄 Hallucinations start with a wrong link and can snowball. Always check!
- 🏋️ AI is trained by guessing and checking, then learning from people. Good teachers make good AI.
- 🎲 Every AI answer is a statistical choice: words picked by chance from patterns. Kind words are not real feelings. And watch out for the Yes-Bot: AI often agrees with you just to please you.
🌟 Chatty's golden rules for using AI
Read each rule. Tick it to make a promise. Make all 7 promises to unlock your badge!
0 / 7 promises made
🔒 Make all 7 promises to unlock your badge!