Part Two — How to Read a New Model
Part Two plate — yellow laid in. Art by Douglas John Brownfield.
PART TWO — HOW TO READ A NEW MODEL
---
Chapter 10 — The Five Questions
Models are announced constantly, and the announcements are written to impress. A launch page leads with the largest number it can find, a name chosen by a marketing department, and a list of records the model has set. None of that tells you what the model is. Fortunately, sorting a new model does not take long. Five questions, asked in order, will place almost any model on the map in Part One — and the questions stay useful after the launch post is forgotten, because they ask about the model's shape rather than its publicity.
1. What does it take in, and what does it give back?
This is the first question because it does most of the sorting. A model that takes in text and gives back text is a talker, whatever its name suggests. A model that takes in words and gives back pictures is an image model. A model that takes in sound and gives back text is doing transcription. The exchange — what crosses the boundary in each direction — decides the type before anything else about the model matters.
In practice, read the input list the way you would read the label on a tool. A short list means a specialist. A long list means the model is multimodal, a native of more than one medium. Google's Gemini 4 Argon, the flagship in this book's November 2026 snapshot, accepts text, images, video, audio, and PDF files as input. That single fact places it: it is not a plain talker with attachments but a model built from the start to work across media. The opposite shape is just as informative. A model that takes in text and gives back numbers — ordered lists of numbers that stand for meaning — is an embedding model, one of the librarians. It will never write you a paragraph, and it is not supposed to.
2. Does it answer, or does it act?
A model, by itself, answers. It receives input and returns output, and then it stops. Acting — using a tool, changing a file, booking, buying, running a program, checking the result and going again — requires a loop around the model: tools it can call, memory of what it has done, and permission to proceed. That loop is a layer built on top of a model, not a kind of model, and the distinction is practical rather than pedantic.
When a product is described as an agent, this question separates the two parts so you can judge each one. The model inside may be an ordinary flagship talker. The acting belongs to the harness around it. OpenAI's agent mode, for example, was superseded in September 2026 by a system of always-on agents with their own cloud computer — the models underneath changed on their own schedule, while the layer above them was rebuilt entirely. If a demonstration shows software being operated on your behalf, you are watching a layer at work. Ask which model is inside it, and ask separately what the layer is permitted to do. Those are different questions with different risks.
3. Where does it run?
Some models run in a data center and reach you through an app or an API. Others run on the device in your hand. The answer changes what the model can offer you: a model on your device works without a connection, keeps its input on the device, and answers at the speed of the hardware you own. A model in the cloud can be far larger, and it changes whenever its maker updates it — including, occasionally, being replaced or retired underneath the products built on it.
Size words in a model's name are usually a clue to its address. The small types in this book — Google's Gemma line, Microsoft's Phi line, Apple's on-device models — are built to live on phones and laptops. Apple's current generation is explicit about the arrangement: a compact core model runs on the device, sized in the low billions of parameters, with a larger sparse model beside it that wakes only a fraction of itself for any given task. When a name carries a word like mini, nano, or lite, the second question to ask is whether it names a smaller cloud model or a model meant to move in with you. The two are not interchangeable, and the difference shows up the first time the connection drops.
4. Can you have the weights, and under what license?
The weights are the model itself — the numbers that everything else is built around. If the maker releases them, you can download the model, run it on your own hardware, and keep the exact version you tested. If the maker does not, you have access to a service, and the service can change.
The word open is where careless reading costs the most, because in current practice it covers very different arrangements. DeepSeek's flagship models are released under the MIT license, which permits nearly any use, including commercial use. Meta's Muse Glimmer is released under Apache 2.0, a similarly permissive license. Ideogram shows why the license must be read rather than assumed: its 4.0 model was released as downloadable weights under a non-commercial license, while its 4.5 flagship remained closed. Same company, same year, two opposite answers to this question. Other releases carry territory restrictions or vendor-specific terms. So the question is never simply whether a model is open. It is: can you have the weights, and what does the license that comes with them actually permit?
5. What is it dated?
Every claim about a model belongs to a month. Lineups turn over inside a year: versions are superseded, tiers are renamed, and models are retired on schedules that their own makers publish. A statement that was accurate when it was written — in a review, a tutorial, or an earlier edition of a book — may describe a model that no longer exists in that form, or at all.
The clearest case in this book's snapshot is Sora. OpenAI's video model was, through 2025, the most prominent name in its kind. The Sora app shut down on April 26, 2026, and the models were removed from the API on September 24, 2026, with no replacement; as of November 2026, OpenAI offers no video model. Nothing about the kind changed — video models remain a type, with other makers' models on the shelf. What expired was the name, on a date. This book handles the problem by carrying its snapshot openly: the examples in Part One are stated to be the roster as verified in November 2026, and the kinds are the part meant to last. A reader can apply the same discipline to anything written about models, including this book. Note the date of the claim, and weigh it accordingly.
The five questions, together
When the next model is announced, ask:
- 1. What does it take in, and what does it give back?
- 2. Does it answer, or does it act?
- 3. Where does it run?
- 4. Can you have the weights, and under what license?
- 5. What is it dated?
The answers will not tell you whether the model is good. That is a separate judgment, and the next two chapters take it up — first the names and numbers attached to a model, then the scores it is advertised with. But the five questions will tell you what kind of thing you are looking at, and most confusion about AI models is, at bottom, a failure to sort before judging.
---
Chapter 11 — Names and Numbers
A model name looks like a label. Read it as an advertisement that happens to contain some facts. Every name on the November 2026 shelf was assembled by a marketing department out of real parts — a family name, a generation number, a tier word, sometimes a size, sometimes a date. The parts are worth decoding, because each one tells you something true. The warning comes first, though, because it governs everything after it: the name is the least reliable information about a model. It was chosen to sell the model, not to describe it. Chapter 10's five questions — what kind is it, what goes in and out, open or closed, how big and what does it run on, who made it and when — will tell you more in a minute than the name tells you at all. Decode the name anyway. Just never let it outrank the questions.
The anatomy of a name
Most names on the shelf are built from the same kit, in roughly this order: family, generation, tier, size, date. Not every name carries every part, and vendors arrange them differently, but once you see the kit you can take almost any name apart.
The family name is the vendor's brand for a line of models: GPT, Claude, Gemini, Llama, Qwen, Mistral. It tells you who made it and which shelf it sits on, and it is the most durable part of the name — families last for years while everything after them churns. It tells you nothing about capability. A family name is a surname, not a description.
The generation number marks the family's big revisions: Gemini 4, Llama 4, Suno v6, Kling 4.0. Treat it as an edition number on a book. A higher number means a later, usually stronger revision of the same line — and nothing more precise than that. Generation numbers are not comparable across families (one vendor's 4 is another's 6, and both numbers were chosen by the vendor), and they are not even consistent within a family: the decimal after the generation — the 5.5 in Claude Opus 5.5, the 3.1 in Veo 3.1 — marks a smaller step, but how much smaller is the vendor's private judgment. Anthropic's shelf in November 2026 carries Claude Fable 5.1 above Opus 5.5 above Sonnet 5.5: three models whose numbers differ by tenths, released within weeks of each other, sitting in a strict capability order that the numbers alone would never tell you.
Then come the tier words, and these deserve their own section, because they are where names mislead most politely.
Tier words rank a family, not the field
OpenAI's current line is the cleanest worked example on the shelf. The GPT-6 family is sold in named tiers: Astra at the top, then Sol, then Terra, with Luna as the high-volume tier. None of those words describes what the model does. They are ranks — a ladder inside one family, in descending order of capability and price. Google does the same thing with different words: Gemini 4 Argon is the generally available flagship, and Gemini 4 Carbon, announced as the stronger model, is gated to government and trusted testers first. Anthropic's ladder is Fable, Opus, Sonnet, Haiku — poetry instead of astronomy, same device.
The older tier vocabulary — Pro, Max, Ultra, Flash, Turbo, Mini — works identically, and the shelf is covered in it: DeepSeek V4-Pro, Qwen3.8-Max, ElevenLabs' Flash v2.5, Kling's Turbo tier, the mini and nano models every vendor parks at the cheap end. Here is the rule that makes all of these words safe to read: a tier word is a comparison inside one family and nowhere else. "Max" means the top of that vendor's current line. It does not mean the strongest model of its type, it does not mean stronger than another vendor's plain model, and it does not stay true — when the next generation ships, last year's Max becomes this year's middle. "Flash" and "Turbo" promise speed relative to their siblings, not speed relative to the field. "Ultra" at one company can sit below "Pro" at another, and both companies are being honest by their own private scales. The words are not lies. They are local currency, and they do not exchange.
The numbers in open names
Closed models hide their size; open-weight models often print it in the name, and those numbers reward a closer read. When a name carries a figure like 3B or 9B, it is a parameter count — billions of parameters, roughly the model's size, and a fair guide to what hardware it needs. Alibaba's small Qwen3.5 series runs 0.8B, 2B, 4B, 9B, and Meta's small Llama 3.2 models are named 1B and 3B: in both cases the number is the point of the model, because it tells you the thing runs on a phone or a modest laptop.
Mixture-of-experts models carry two numbers, and confusing them is the most common honest mistake in reading a name. An MoE model holds a large total pool of parameters but activates only a fraction of them for any given piece of work. Mistral Large 4 is the shelf's clear example: 1T total, 52B active — one trillion parameters on the shelf, about fifty-two billion awake at a time. Alibaba's open Max-class release makes the same distinction inside the name itself: Qwen3.8-2.4T-A95B, total first, active after the A. Apple's on-device line shows why the distinction matters in practice: alongside a plain 3B model sits a 20B sparse model that activates only 1–4B parameters at a time, which is how something with 20B in its name runs on a phone. Read both numbers when a name offers both. The total tells you the model's capacity and its storage weight; the active figure tells you its running cost. A name that offers only one number is telling you only half of that.
Dates, snapshots, and the same name twice
The last part of the kit is the date, and it cuts two ways. Some names and identifiers carry dates outright: Anthropic's computer-use toolset is versioned computer_toolset_20260801 — the date is in the name, August 1, 2026, so a developer can pin an exact, unchanging thing. API model names commonly work the same way, with a dated snapshot suffix marking one frozen version. A dated name is the most honest name on the shelf: it promises that the thing behind it will not change.
The undated name promises the opposite. A bare name in an API is frequently an alias — a pointer the vendor can re-aim. OpenAI's gpt-5.6 alias routes to GPT-5.6 Sol; the name you call is not a model but an address, and what answers at that address is the vendor's current choice. The same effect arrives without aliases, too: a family word like Opus has now covered several distinct models across the 4 and 5 generations, each quietly replacing the last under a word that stayed still. So the rule from Chapter 10 applies with full force here — who made it, and when is a question about a specific dated model, never about a name. When a name matters to your work, find its date. When it has no date, assume it is a moving target and plan accordingly.
The standing rule
None of this makes names useless. The family tells you the maker, the generation tells you the edition, the tier word tells you the rank at home, the printed numbers — read in pairs, on MoE models — tell you the size honestly, and a date, when present, pins the whole thing down. What a name cannot do is compare across vendors, promise a capability, or substitute for the five questions. Names are marketing first and description second, written by the seller, in the seller's private vocabulary, at the moment of sale. Read them the way you read a label on a market stall: useful for finding the stall again, no substitute for picking the goods up. The questions are the picking up.
---
Chapter 12 — Benchmarks Without Tears
Every model launch arrives with numbers. A new model is announced, and within the first paragraph there is a score: so many points on this test, a few points ahead of a rival on that one. The numbers look like verdicts. They are not. They are measurements, taken under particular conditions, and they become useful only when you know what was measured, when, and by whom. This chapter is about reading those numbers calmly.
What a benchmark is
A benchmark is a fixed test that many models take. The test might be a set of mathematics problems, a collection of coding tasks, a battery of questions drawn from school and professional examinations, or a series of instructions that human graders score against a rubric. The point of fixing the test is comparison: if every model answers the same questions under the same rules, the results can be set side by side.
That is all a benchmark score is. It is a record of how one model performed on one test, on one date, under one set of settings. Change any of those three — the test, the date, the settings — and the number can change with them. A score is not a property of a model in the way its size is. It is closer to a race time: real, recorded, and meaningful only alongside the course and the conditions.
What leaderboards measure, and what they cannot
A leaderboard gathers benchmark results into a ranking. Rankings are easy to read, which is exactly their danger. A leaderboard can tell you which models performed best on the tests it tracks, as those tests were run. It cannot tell you which model will write the clearest email for you, follow your house style, handle your spreadsheet, or stay accurate across a long document in your field. Your work is not on the test.
Two kinds of leaderboard are worth knowing apart. Some scores come from tests a laboratory runs on its own models — or on rivals' models — and publishes itself. Others come from standing public boards that run comparisons continuously and date what they publish. LMArena is the best known of the public kind: it pits models against each other in blind comparisons and ranks them by how people vote. Artificial Analysis is another standing public source, tracking models across quality, speed, and price, with its methods and dates published alongside the results. Neither board settles which model you should use. Both are useful for the same reason: they are third parties, they work to a stated method, and their results carry dates, so you can see whether a ranking is current or a season old.
Settings matter as much as the test itself. Many models can be run at different levels of effort — more time spent working through a problem, more attempts made before answering — and higher effort usually means a higher score at a higher cost. Two results on the same test are comparable only if the settings were comparable, and launch materials do not always say what settings were used. When they do not, read the score as the model's best case, not its everyday one.
Why vendors quote the tests they win
A launch post is an advertisement as well as a report. The laboratory chooses which benchmarks to feature, and it features the ones its model won. A model that leads on coding tests will be introduced as a coding model with the coding scores up front; a model that leads on examination questions will arrive wearing those. Nothing about this is dishonest by itself. The tests are usually real, and the scores are usually accurate as far as they go. But the selection is the vendor's, and a reader who sees only the selected tests has seen the model's best subjects, not its whole report card.
The tell is absence. When a launch post is silent about a well-known test that earlier launches from the same company quoted proudly, the silence usually means the new model did not win it. You cannot learn a fact from an absence, but you can learn where to look next.
Contamination and teaching to the test
There is a second reason to hold benchmark scores loosely. A benchmark only measures well if the model has not seen the test before. Test questions circulate. They are quoted in articles, posted in forums, and folded into the enormous bodies of text that models are trained on. When a model's training data includes the test — or close copies of its questions — the model is no longer solving the problems cold. It is, in part, remembering them. This is called contamination, and it inflates scores in a way no reader can detect from the number alone.
Alongside contamination sits a plainer habit: teaching to the test. Once a benchmark becomes the industry's scoreboard, laboratories tune their models toward it, because the scoreboard is what gets quoted. The model genuinely improves at that test. Whether it improves at everything the test was meant to stand for is a separate question, and usually nobody runs it. Old benchmarks also wear out: as models cluster near the top of a test, the test stops separating them, and the field moves on to a harder one. A benchmark's age is therefore part of its meaning. A high score on a saturated test from several years ago says less than a middling score on a current one.
The reader's working method
None of this requires you to ignore benchmarks. It requires you to read them the way you would read any claim: with four questions.
Check the date. Models turn over in months. A score from a year ago describes a model that may have been revised, replaced, or retired since. Dated results can be weighed; undated ones cannot.
Check who ran it. A result published by the model's own maker is a claim. The same result confirmed by a public board or an independent evaluator is evidence. Both can be true; they do not carry the same weight.
Prefer the test closest to your task. If you need a type of model for translation, a translation benchmark matters more than a mathematics one, however famous. If none of the standard tests resembles your work, the honest test is a small one of your own: a handful of real tasks, run through each candidate, judged by you.
Treat any undated score as advertising. A number with no test name, no date, and no source is not information. It is decoration on a claim, and it should move your judgment not at all.
Read this way, benchmarks return to their proper size. They are a first filter — a quick way to see which models are in the leading group of their kind, and which claims deserve a closer look. The final judgment belongs to the only test that includes your work: yours.
---