Most AI glossaries fail for the same reason phone numbers are hard to memorise: twelve unrelated items, no structure, nothing holding them together. You read the list, you nod, and a week later you can recall three.
So this one is built differently. The same twelve essential AI terms, explained in the order an AI system actually experiences them — and carried by a single image that makes them stick.
The image is a student. A model is born, reads everything, gets schooled, gets examined, gets hired. Every term below is one moment in that life. Learn the story and the vocabulary comes free.
Here is the whole thing in one sentence, which you can come back to once you have read the article:
A student (LLM) reads the world in bites (tokens) and keeps everything it learned as numbers (weights). It graduates once (training) and then works forever (inference). It takes a specialist course (fine-tuning), practises with a coach (reinforcement learning), is marked on questions kept out of the revision notes (validation loss) and — like every exam-taker — guesses rather than leaving a blank (hallucination). Then it coaches a cheaper apprentice (distillation), sits open-book exams (RAG), is asked to show its working (chain of thought), and finally takes the keyboard (coding agent).
Four stages, three terms each. Let’s walk through them.
Stage one — The student: what an AI is made of
1. LLM (Large Language Model)
A large language model is a system trained to predict what most plausibly comes next in a sequence of text. That is the entire engine. Answering questions, writing code, summarising a contract, running an agent — all of it is that one behaviour applied at enormous scale.
The crucial consequence is what it is not: there is no database inside it, no index, no lookup table. When a model “knows” the capital of Bangladesh, that knowledge exists as a pattern spread across billions of numbers, not as a stored record it retrieves.
Remember it as: a prediction engine, not a library.
2. Token
A model does not read words. It reads tokens — fragments the model’s tokeniser carved out of text because those fragments were frequent in its training data. Unbelievable might arrive as un + believ + able. Sometimes a token is a whole word, sometimes a punctuation mark, sometimes three letters.
Two practical consequences follow, and they are the reason this term appears on every glossary:
- You are billed per token, on input and output. Your system prompt, retrieved documents and the entire conversation history are re-sent on every single call. A short question sitting on top of a long conversation is not a cheap request.
- Tokenisation is not language-neutral. Scripts that were less common in training — Bengali, Hindi, Arabic, Thai — are typically split into more tokens for the same meaning. The same paragraph can cost several times more in Bengali than in English. If you build for a non-English market, measure this with your real prompts before you price anything.
Remember it as: it reads in bites, not words — and you pay per bite.
3. Weights
Weights are the billions of numeric parameters inside a model that were adjusted during training. They start out random. Training nudges each one, over and over, until the model’s predictions match the patterns in its data. When training stops, those settled numbers are the model. Everything it learned is stored there.
This is also why you will keep hearing “open weights.” It means a lab published those parameters, so anyone can download the model and run it on their own hardware — which turns a technical detail into a procurement and compliance decision. Data that cannot leave your jurisdiction can still be processed by a model whose weights you host yourself.
Remember it as: everything the education left behind, stored as numbers.
Stage two — The schooling: how an AI got good
4. Training vs inference
Training is school: an enormous, one-time process where the weights are set. Inference is the job: the model taking an input and producing an output. Training costs a fortune and happens once. Inference costs a little and happens every time someone presses enter.
The practical implication catches almost everyone new to this. A model does not learn from your prompts. Nothing you type at inference time changes a weight. It has no memory between calls. Anything that behaves like memory — chat history, a user profile, a knowledge base — is context that your system deliberately re-sends on every request.
That single fact explains the shape of most AI engineering work: you rarely change the model, so your job is to change what you put in front of it.
Remember it as: training is school, once. Inference is work, forever.
5. Fine-tuning
Fine-tuning takes an already-trained model and trains it further on a narrower set of examples, adjusting the weights toward your particular use.
What it excels at is behaviour: output format, tone of voice, a domain’s phrasing conventions, a consistent classification boundary. What it handles poorly is facts, because facts change and weights do not. Bake a price list into weights and you own a retraining schedule forever.
The order I use on client projects, and recommend without much hesitation:
- Prompt — free, instant, fixes more than people expect.
- Retrieve — for anything that changes (see RAG below).
- Fine-tune — for behaviour that prompting cannot hold reliably.
Remember it as: fine-tuning changes how it speaks, not what it knows.
6. Reinforcement learning
Reinforcement learning shapes a model through reward instead of examples: it produces an output, the output is scored, and the weights shift toward whatever scored well. Repeat at scale. This is how a raw text predictor becomes a helpful assistant, and how reasoning models were sharpened — particularly in domains where the reward can be checked automatically, like a unit test passing or a maths answer matching.
The dog-training comparison everyone reaches for is fine, with one addition worth carrying: the learner optimises the measured signal, not your intention. Reward answers that humans rate highly and you get a model that is agreeable, which overlaps with being correct but is not the same thing. Reward a verifiable outcome and you get genuine capability gains in that narrow domain.
This matters the moment you build your own evaluation loop — an LLM-as-judge pipeline, a scoring rubric, an automated quality check. Whatever you measure is what you will get.
Remember it as: it learns what you reward, not what you meant.
Stage three — The report card: how we know it worked
7. Validation loss
During training, the model is regularly tested on examples deliberately kept out of its training data. Validation loss is the error score on that held-out set. Falling validation loss means the model is generalising. Falling training loss with rising validation loss means it is memorising — that is overfitting, and this metric is how you catch it before wasting a run.
Two qualifiers make the difference between quoting the term and understanding it. Lower is better only if the held-out data resembles what your users will actually send; a validation set collected under conditions your production traffic does not share gives you a beautiful curve and a disappointing product. And loss measures prediction error, not usefulness. Loss can improve while the thing your business cares about stays flat.
Remember it as: the practice exam, marked on questions kept out of the revision notes.
8. Hallucination
A hallucination is a confident, fluent, false statement. Everyone knows the symptom. The interesting part is the cause, because it determines how you fix it.
It is tempting to assume the model simply lacked the information. Sometimes it did. But research published in 2025 (Why Language Models Hallucinate) points at something more structural: models are trained and graded in a way that rewards a confident guess over an admission of uncertainty. Think of a multiple-choice exam. A blank scores zero with certainty; a guess might score. A student optimising for marks always guesses — and so does a model optimised to score well on benchmarks.
Read that way, the engineering response changes completely. You do not fix a guessing incentive by adding data; you fix it with design:
- Ground the answer in retrieved source material rather than parametric recall.
- Give it an exit — a prompt and a schema that make “not found in the provided documents” a valid, expected output.
- Constrain the output — structured formats leave less room for fluent invention.
- Verify separately — a second pass, a different model, or a deterministic check on anything that matters.
Remember it as: it guesses because a guess scores better than a blank.
9. Distillation
Distillation trains a small model to reproduce a large model’s behaviour. The large model generates high-quality outputs; the small model learns from them and ends up doing much the same job at a fraction of the cost and latency. This is a major reason capable AI now runs on a phone, and why “small model” no longer implies “weak model.”
The honest boundary: a distilled model is faithful inside the distribution it was distilled on and degrades outside it. Evaluate it against your awkward cases, not your demo script.
Remember it as: the graduate coaching a cheaper apprentice.
Stage four — The job: how an AI works for you
10. RAG (Retrieval-Augmented Generation)
RAG is the pattern behind most useful enterprise AI. Instead of hoping the model absorbed your information during training, you search your own content at question time, pull the relevant passages, and hand them to the model along with the question. A closed-book memory test becomes an open-book exam.
Why it dominates: your facts stay outside the model, where you can update them, permission them and audit them. Change a document and the next answer changes. No retraining.
Its performance depends entirely on the retrieval step, which is where nearly all real failures live. When retrieval misses, you do not get an obvious fabrication — you get a plausible answer with a citation attached, which is harder to catch. The five failure points, in the order I encounter them:
- Chunking that separates a fact from the context that gives it meaning.
- Embeddings that surface topically similar but factually irrelevant passages.
- No reranking, so the best passage sits at position fourteen and never reaches the model.
- A stale index — the document was updated, the vector was not.
- No abstention path, so when retrieval returns nothing useful the model falls back on parametric memory and invents.
If you take one debugging habit from this article: measure retrieval quality separately from answer quality. Most “the AI is wrong” tickets are retrieval tickets.
Remember it as: an open-book exam — it isn’t recalling your documents, it’s being handed them.
11. Chain of thought
Chain of thought is prompting a model to work through a problem in explicit steps before committing to an answer. On multi-step tasks — arithmetic, logic, planning, anything with dependencies — it measurably improves results, because each intermediate step becomes context for the next. On simple lookups and classifications it mostly adds latency and cost.
One nuance worth holding, especially if you are building audit trails: the reasoning a model prints is generated alongside the answer, not extracted from the process that produced it. It is a genuinely useful debugging artefact and a plausible justification. Treat it as a window, not a recording.
Remember it as: showing your working — useful, but it’s an explanation, not a recording.
12. Coding agent
A coding agent does not suggest code for you to accept. It reads the repository, writes the change, runs it, reads the error, fixes it, runs the tests, and iterates — with limited supervision. The gap between autocomplete and agent is the gap between a colleague who tells you what to type and one who takes the keyboard.
The capability is real and it is the fastest-moving item on this list. The constraint has moved accordingly: it is no longer generation, it is review. And agent reliability tends to fall with session length rather than task difficulty, because a long session accumulates stale file contents, abandoned approaches and its own earlier mistakes, all weighted as equally true context.
What works in practice, from running these daily:
- Scope one task per session, then start a fresh one.
- Give it a way to check itself — tests, a type checker, a linter. Verifiable feedback is worth more than a better prompt.
- Review in a new session rather than asking the same one to audit its own work.
- Read the diff. An agent that writes ten times faster also writes plausible-looking wrong code ten times faster.
Remember it as: it doesn’t suggest, it does — and someone still has to read the diff.
The recall table
Cover the right column and work down. If you can reconstruct the middle column from the metaphor, you know the term.
| Term | What it is | Remember it as |
|---|---|---|
| LLM | Predicts the next chunk of text at scale | A prediction engine, not a library |
| Token | The text fragment a model reads and is billed on | It reads in bites, and you pay per bite |
| Weights | The learned parameters that constitute the model | What the education left behind, as numbers |
| Training vs inference | Setting the weights, versus using them | School once, work forever |
| Fine-tuning | Further training on narrow examples | Changes how it speaks, not what it knows |
| Reinforcement learning | Shaping behaviour through reward | It learns what you reward, not what you meant |
| Validation loss | Error on deliberately held-out data | The practice exam from unseen questions |
| Hallucination | Confident, fluent falsehood | A guess scores better than a blank |
| Distillation | Small model trained on a big model’s output | The graduate coaching a cheaper apprentice |
| RAG | Retrieving your documents at question time | An open-book exam |
| Chain of thought | Reasoning in explicit steps | Showing the working |
| Coding agent | AI that writes, runs, tests and fixes code | It takes the keyboard |
Where to go from here
Twelve terms, four stages, one student. Learned in that order, they stop being vocabulary and become a mental model — and a mental model is the thing that lets you ask a good question about a system you have never seen before.
If you want one habit to carry out of this article: whenever an AI system misbehaves, ask which stage it belongs to. Wrong facts are usually stage four — a retrieval problem, not a model problem. Wrong tone or format is stage two. A great metric with a disappointing product is stage three. Naming the stage is most of the diagnosis.
FAQ
What are the most important AI terms to learn first?
Token, inference, hallucination and RAG. Between them they explain most of what you will pay for and most of what will go wrong. The other eight are easier once those four are solid.
What is the difference between RAG and fine-tuning?
RAG supplies information at question time, keeping your facts outside the model where they can be updated and audited. Fine-tuning adjusts the model’s weights to change its behaviour, tone or output format. Information that changes should be retrieved; behaviour that must be consistent can be fine-tuned. Many production systems use both.
Why do AI models hallucinate?
Because training and evaluation reward a confident guess over an admission of uncertainty, much as a multiple-choice exam rewards guessing over a blank answer. Missing information contributes, but grounding, structured output and verification address the problem more effectively than simply adding data.
Does chain of thought always improve accuracy?
No. It helps on tasks with multiple dependent steps and adds cost and latency on simple ones. The printed reasoning is also an explanation generated alongside the answer rather than a faithful record of the process.
Why does my AI bill grow faster than my usage?
Because you pay for input and output tokens on every call, and system prompts, retrieved context and conversation history are re-sent each time. Non-English text usually consumes more tokens for equivalent content, which compounds the effect.
Do I need to understand the maths to work with AI?
No. Every term on this list can be used correctly from the mental model alone. The maths matters if you train models; the concepts matter if you build with them, buy them or govern them.