Douglas B. Books

What Kind of AI Is That? · read online

Part Three — Choosing

Part Three plate — the finished piece. Art by Douglas John Brownfield.

PART THREE — CHOOSING

---

Chapter 13 — Match the Kind to the Job

Most people choose a model the way they choose a phone: by the ranking at the top of a chart. That is backwards. A leaderboard answers the question "which model scored highest on a test someone else designed?" The question that matters is narrower and plainer: "what work am I handing over, and what kind of model does that work?"

So start with the job, stated in ordinary words. Not "I need AI." Something like: draft a letter. Summarize a long report. Find the passage in my own library that mentions a contract date. Make a picture for a page. Read a chart and tell me what changed. Watch a counter and speak up when it moves. Each of those sentences already points at a kind, because each one fixes what goes in and what comes out. Text in and text out is a talker. A large library in and a short list of locations out is a librarian. Privacy-sensitive material that should never leave the room is a small on-device model. A description in and a picture out is an image type. The match is made by the shape of the work, before any brand name enters the conversation.

Worked examples, at the family level, as the roster stands in November 2026:

• Drafting and summarizing belong to the large language models, the talkers. This is the most served job on the shelf: the GPT-6 family, Claude's 5.5 family, Gemini 4 Argon, and the open-weight talkers from Mistral, DeepSeek, and Qwen all do it well. For ordinary drafting, the differences between current flagships are smaller than the differences between a clear request and a vague one.
• Finding things in a large collection belongs to the embedding and retrieval models — the librarians. A talker asked to "find it" in ten thousand documents is being used as a very expensive librarian and will sometimes invent the shelf it claims to have checked. The librarian kind — OpenAI's text-embedding-3 pair, the Voyage 4 family, the open BGE-M3 — turns the collection into something searchable first; the talker then works on what was actually found.
• Private material belongs on the device. Medical notes, family finances, a diary, unreleased work: if the content should not travel, the model should not either. The small on-device types exist for exactly this — Gemma 4 under the Apache 2.0 license, Microsoft's Phi-4 line under MIT, Apple's on-device models, the small Qwen series. They are less capable than a flagship talker and that is the correct trade: the job's first requirement was privacy, and capability was second.
• Pictures belong to the image types: Midjourney, OpenAI's GPT Image line, Google's Gemini-native image family, the FLUX family. A talker with image output bolted on is a different tool from a model built for pictures, and the difference shows in editing and consistency across a set.
• Reading a chart, a form, or a photographed page belongs to the multimodal talkers. As Chapter 6 recorded, seeing is now a property of the flagships rather than a shelf of its own: Gemini 4 Argon takes text, image, video, audio, and PDF natively, and its peers do the same. The matching point is simply to check that the chosen talker accepts the input the job actually has — a scanned document is not text until some model can see it.
• Holding a conversation belongs to a talker too, but latency picks the tier. A spoken or live exchange fails if each answer takes ten seconds, however good the answer is. The lighter, faster members of a flagship family — or a small model for narrow exchanges — serve conversation better than the largest model answering late.
• Hard problems are not a separate purchase. As Chapter 2 recorded, reasoning is no longer a kind of its own; it is a mode of the flagship talkers, usually an effort setting. The matching move is not to shop for a "reasoning model" but to raise the effort on the talker already chosen — and only for work that repays slow thinking: proofs, plans with many constraints, code that must be right. For a summary, high effort buys delay, not quality.
• Watching a counter — a price, a gauge, a status page — is a job of patience, not power. It wants a small, cheap model that can look often, or a librarian-style check against a threshold, with a talker called in only when something actually moved. Running a flagship continuously to watch a number is paying for eloquence that the job never uses.

Underneath all of these sit three costs, and they trade against each other in a fixed pattern. A bigger model is generally slower to answer, more expensive per answer, and further away — it runs in someone else's data center, under someone else's terms, on a connection that can fail. A smaller model is faster, cheaper, and can be near or on the device, at the price of capability. There is no direction that wins on all three. "Best" without a job attached is a word without a referent; the honest question is which of the three costs the job can afford and which it cannot.

One caution completes the method. Prefer the kind over the vendor, and keep the work portable. A model name is a product, and products are withdrawn. Sora is the standing example in this book: the most publicized video model of its period was shut down in 2026 — the app in April, the API in September — with no replacement, and every workflow built on that name alone had to be rebuilt or abandoned. Workflows described in terms of kinds — "a video type that takes a still and a prompt" — survive the withdrawal of any one name, because another member of the kind can be put in its place. Names are how models are sold. Kinds are how work is planned.

Before choosing, run the short list:

1. 1. State the job in one plain sentence. If the sentence will not come, the choice cannot either.
2. 2. Name what goes in and what comes out. That pair selects the kind.
3. 3. Ask where the material may travel. If it may not leave, the answer is on-device, whatever the leaderboards say.
4. 4. Ask how often the job runs. Rare and hard argues for a flagship with effort raised; constant and simple argues for something small.
5. 5. Ask what happens if this model is withdrawn next year. If the honest answer is "we start over," describe the job in kind terms now, while the description is cheap.

The next chapter takes up what happens when one kind is not enough — when a job needs a finder, a drafter, and a checker working in sequence, held together by the layer this book set aside in Part One.

---

Chapter 14 — The Layer: Loops, Tools, and Models in Combination

This book has sorted models into kinds: talkers, image models, video models, and the rest. One phrase in common use names something real that does not belong on that list, and this closing chapter exists to say where it belongs instead. The phrase is loop model. People use it for systems that do not merely answer and stop — systems that take a task, act on it, look at what happened, and act again, sometimes for hours. The thing they are pointing at exists. It is simply not a type of model. It is a layer built around one, and the distinction matters for the same reason every distinction in this book matters: if you sort it wrong, you will shop for it wrong, price it wrong, and blame the wrong part when it fails.

A model, on its own, does one thing: it takes input and returns output. A talker given a question returns text. It does not open the file the answer depends on, it does not check whether the file changed since yesterday, and it does not notice that its first answer was wrong. Everything that turns a model into a worker is supplied by software around the model. That surrounding software is called a harness, and a system made of a model plus a harness is what the trade usually calls an agent. The model inside an agent is one of the kinds this book has already covered — most often a talker, sometimes a multimodal model. The loop is not in the model. The loop is the harness's way of calling the model, again and again, until the work is done or a limit is reached.

A harness has four working parts, and each one can be described plainly.

The first is the loop itself: act, observe, adjust. The model proposes an action — run this search, open that page, write this draft. The harness carries the action out and brings back the result. The model reads the result and proposes the next action, which may be a correction of the last one. A single pass through that cycle is a turn. A useful agent is mostly this cycle, repeated with judgment about when to stop: the task is finished, the evidence is sufficient, or further attempts are producing the same failure and a person should be asked.

The second part is tools. A tool is a defined action the harness will perform at the model's request: read a file, query a database, search the web, send nothing yet but prepare a draft, run a calculation, operate a browser. Tools are where a model's statements meet the world. They are also where the harness keeps control, because every tool is a door the harness's builder chose to install, with a defined shape — this tool can read but not write, that one can prepare a message but not send it.

The third part is memory. A model remembers nothing between calls; whatever continuity an agent has is stored by the harness and handed back to the model as context: the task list, the notes from earlier turns, the record of what has already been tried and what it produced. When an agent appears to remember your preferences from last week, it is the harness retrieving them, not the model recalling them.

The fourth part is permissions, and it is the part that decides how much any of this should be trusted. Permissions state what the system may do on its own, what it may do only with a person's approval, and what it may not do at all. A mature harness treats sending, publishing, paying, and deleting as a different class of act from reading and drafting, and it routes the first class through a human being.

As of November 2026, the layer is not a research curiosity; it is shipping product, and it is moving at least as fast as the models are. Anthropic's computer use — a model seeing a desktop through screenshots and driving it with mouse and keyboard — has graduated from beta to general availability on its API, packaged as a dated toolset of seventeen member tools, alongside a separate toolset that acts on web pages directly rather than through pixels. OpenAI's agent mode, the task-at-a-time agent it shipped in 2025, was superseded in September 2026 by a product called dots: always-on agents, each with its own cloud computer and its own browser, intended to keep working toward a person's goals between conversations rather than starting cold at each request. Google's path ran the other direction. Its named agent experiment, Project Mariner, was shut down on 4 May 2026, and the capability did not disappear — it dissolved into Gemini Agent inside the Gemini app, and into agent features spread through Google's other products. The experiment ended because the layer had become a feature, not because it had failed.

The layer also has an open shelf, and it matters for the same reason open weights matter: a harness you can read is a harness whose permissions you can verify. Browser Use, an open-source library that lets almost any model drive a real browser in a loop, is released under the MIT license and has become standard equipment that other agent products build on. OpenHands, which began as an open coding agent, has grown into Agent Canvas — a self-hosted control center that runs many different agents, from several different makers, under one set of rules — also MIT licensed. That second fact is the layer's clearest self-portrait in 2026: the leading open agent project now describes itself as a panel for controlling other agents. Harnesses are being stacked on harnesses, which is only possible because the layer is a real, separable thing.

In practice, most serious work at the layer is not one model in a loop but several kinds in a chain. A retriever finds the relevant material and returns it with its sources. A talker drafts from that material. A checker — a second model call, a rules engine, or a person — tests the draft against the sources and against the facts it can verify independently. The loop carries the whole sequence, keeps the record of what was checked, and stops where it is told to stop. Each kind in the chain is doing the job its chapter in this book describes; the harness is what makes their separate outputs add up to finished work.

Consider one ordinary arrangement, described by its method only. A one-person operation — publishing, correspondence, a schedule of recurring products — is run day to day by one person working with an AI collaborator. Nothing in the arrangement is exotic. The collaborator works from written instructions and a standing record, so its context does not depend on anyone's memory of last month's conversation. Work runs on schedules: drafts are prepared before they are needed, never at the deadline. Every draft passes a verification gate before it reaches the person — figures are re-derived from their sources, names and dates are checked against the record, and anything that cannot be confirmed is flagged rather than smoothed over. The person approves what goes out; the system does not publish, send, or commit on its own authority. When the person corrects the work, the correction is written into the standing record the same day, so the same mistake is not offered twice. The models inside such an arrangement are interchangeable — a better talker can be swapped in when one appears, and the schedules, gates, and records carry on unchanged. That interchangeability is the practical proof of this chapter's claim: the durable asset is the layer and its discipline, not any single model passing through it.

That is also where the person stands in the loop, and it is the right note to end on. A harness multiplies what a model can do, but it does not decide what should be done, and it does not carry responsibility for the result. Permissions are the formal expression of that fact: the wider the system's freedom to act, the more explicit its limits and its approval points have to be. The kinds of models will keep changing — this book's snapshot will age, some of its shelf entries will melt or be retired, and new types will form that have no chapter here. The layer will change with them. What does not change is the sorting habit this book has tried to teach: ask what kind of thing you are looking at, ask what is model and what is machinery around it, and keep the deciding — and the responsibility — with the person the work is for.

---