Skip to Content

Decision models: not every AI step needs an expensive language model

Gemini Decision 1 picks from fixed options at a fraction of the price, if your process has clear options.
November 17, 2026

Not every step in an AI process needs a language model that writes text. If a ticket, invoice or lead only has to land in a fixed category, a decision model can do it at a fraction of the cost. Google released one on 11 November: Gemini Decision 1. It only pays off if your process already has clear options and you measure on your own examples.

What we came across

We follow a fixed circle of makers and newsletters who work with AI themselves. Latent Space wrote twice this week about this kind of model, which returns a choice, a score or a probability instead of text.

Choosing instead of writing

Gemini Decision 1 does not write an answer, it scores fixed options in a single step. According to Google:

  • Yes or no, multiple choice with 2 to 255 options, or a score on a rubric.
  • Up to 1,000 questions per call, across text, images, audio, video and PDF.
  • A confidence score from 0 to 1 with every answer, so doubtful cases can go to a larger model or a person.
  • 0.10 dollars per million input tokens, nothing for output, 150 to 250 milliseconds per answer.

Our view: that confidence score is the most important part, because it lets you decide when a person takes a look.

Cheap per token, not necessarily per task

Google calls it nearly five times cheaper than Gemini 3.5 Flash-Lite. The independent testing firm Artificial Analysis sees a more mixed picture:

  • On their decision index it scores 66, level with Jev from TypeSafe AI.
  • On documents with several questions it got everything right on 33 percent of the documents; the best competitor reached 63 percent.
  • A million tasks cost 345 dollars, against 156 dollars for Jev. A task took 211 milliseconds on average, against 92 for Jev.

Our view: calculate per task, not per token, as with any model. Here the lowest token price did not give the cheapest task.

Companies already use it, partly still as a trial

Jev came out in September. At the AI Engineer Code Summit, companies showed what they do with it:

  • Glean uses Jev in its product, alongside language models. When the categories change, the model does not need retraining.
  • UiPath had thousands of document chunks ranked tournament-style in a test, in under a second.
  • Workday showed a prototype that fills in a time-off request while the user is still typing.

Our view: mind the difference between production and trial. Only at Glean does it really run, and all figures come from the speakers themselves.

Small model sorts, large model thinks

Cloudflare showed a setup in which fast decision models pick the tool and read log lines, and a large reasoning model only makes the diagnosis. In their demo this went from six calls to one, and from about 40,000 to 1,800 tokens.

Our view: this is the setup we would recommend, because the expensive model then only gets work that really needs thinking.

What you do now

  • Find the choice steps in your process. Where does AI currently classify a ticket, invoice or lead, in Odoo or another system? If the options are fixed, that is a candidate.
  • Make the options clear first. A decision model only chooses from what you give it.
  • Measure on your own examples. Have the model classify a few hundred handled cases and compare with your team. Then you know how often it is right and what it costs per task.

How it works under the hood

A language model writes its answer token by token. A decision model receives the options and calculates the probability of each option in a single computation. Only trust the confidence score once you have seen on your own data that it holds: set a threshold and pass everything below it on. The figures in this piece come from Google, Artificial Analysis and the speakers; treat them as an indication.

This news touches chapters 9, 23 and 24 of the course AI(l) or nothing by Newway. If you follow the course, this subject returns there at a calmer pace.

What you can do with this

Want to know what this means for your business? Plan an introduction: half an hour, no sales pitch, but an honest answer to the question where to begin. Rather read on first? The course AI(l) or nothing is free and starts with the basics.

Written in response to publications by Latent Space on Decision Models Day and Latent Space on Jev.