Jev, LLMs and the gap between confidence and authorization.

Decision models can make agentic AI easier to govern, but only if firms stop treating their probabilities as self-validating. Instead of generating text, they take unstructured state and a set of typed questions and return bounded answers with probabilities that software can branch on directly.

For high-volume decisions inside agents — routing, triage, escalation, action approval, output verification — that interface is a genuine improvement over parsing an LLM’s prose. The governance question is narrower, and harder, than the marketing suggests.

Most of what enterprises need AI to do inside a workflow is not writing. It is deciding. Agentic systems make many small judgments per task, and each one determines what happens next:

• Should this request be approved, or does it need a person?
• Which queue or workflow should handle it?
• Is this agent's proposed action within its authority?
• Does the evidence support the action being proposed?
• Is this behavior anomalous?

Large language models can answer these questions, often well. But they answer them the way they answer everything: by generating text. The software around them must then parse that text, validate it and hope the format holds. When the model is buried several layers deep in a dependency chain, a malformed or unexpected response is not an inconvenience. It is a failure mode.

There is a second problem. LLMs are not designed to say how sure they are. They can be prompted to state a confidence number, but that number is generated after the fact and tends to be overconfident.

Jev, which entered early access on 15 September 2026, is a text-in, typed-decision-out model: you send it state and a set of questions with predefined answer spaces, and it answers all of them in one parallel pass. TypeSafe describes a new architecture, a parallel sampler and a training method it calls Reinforcement Learning for Calibrated Decisions. Every question is one of three types: a yes/no question, a pick-one choice from a defined set, or a position on an ordered rubric.

One difference between them matters more than the write-ups suggest. Only the latter two expose a confidence value, and it is derived from the underlying probabilities rather than produced separately. A yes/no question returns a single probability and nothing else — so governance logic that expects a confidence field on every decision will silently misbehave on exactly the questions most likely to gate an action.

Probability, confidence and calibration get used interchangeably and answer different questions. Probability is how strongly the model supports this answer relative to the alternatives.Confidence is how concentrated its distribution is for a Choice or a Score. Both are properties of the output, established by the model at inference time. Calibration is a property of how that output relates to the world: when the model says 0.8, does the event happen about 80% of the time in our population? A vendor can train for it. Only the deploying firm can demonstrate it.

For a bank the distinction is concrete. A fraud, complaint-severity or policy-breach decision has an observable outcome: the transaction was fraud or it was not, the complaint was upheld or it was not. Calibration has to be measured against that, on the firm’s own population, before any threshold is set.

Calibration is also local. A model can be well calibrated on consumer card transactions and poorly calibrated on commercial wires. It can be calibrated this quarter and drift next quarter as the population or the agent’s behavior changes. A single global model does not imply globally valid thresholds.

And it does not compose. Workflows that ask several narrow questions about the same state and combine the answers in code produce a combined decision with its own error rate, which has to be measured directly rather than inferred from individually calibrated parts.

The hardest governance risk is not a wrong answer on a validated decision. It is a confident answer on a decision nobody validated. Goal-directed agents create paths their designers did not enumerate. When a decision model sits across all of those paths, some of its answers will land in territory where its probabilities have never been checked. Three kinds of uncertainty are in play, and each needs a different control.

1. Model uncertainty. Is the model unsure about this answer?

   Answered by the model itself, through its probabilities. This is the only one a high probability speaks to.

2. Domain uncertainty. Have we shown these probabilities mean anything for this kind of decision, on this population?

   Answered by the firm's validation evidence. A 0.97 on a decision nobody validated tells you nothing here.

3. Governance uncertainty. Is the agent allowed to take this action at all?

   Answered by policy and entitlements, deterministically. No probability speaks to this at any value.

Forced choice makes this worse. Type safety is usually presented as pure upside: a pick-one question cannot return an option outside its set. The flip side is that it cannot decline either. It always names a winner, which is why practitioners are already advising that “none of these” be asked as a separate yes/no question. An out-of-scope case therefore does not produce an error. It produces a well-formed, in-scope answer with a respectable probability, and nothing in the output flags that the question did not really apply.

The natural control is a coverage gate: before an answer can drive a consequential action, check that the decision is registered and the state is within the validated domain. But “is this state within the validated domain?” is itself a judgment, and the obvious tool for it is another question to the same model, asking it to certify its own applicability. That circularity cannot be engineered away entirely. It can be reduced by drawing coverage evidence from signals that do not come from the same model: registration that fails closed on unregistered decision types, structural checks on the state, and novelty signals computed by a different method.

The defensible deployment is as an intelligence layer feeding a separate, deterministic control layer. The model characterizes the situation; code and policy decide what is permitted. Confusing the two is how model confidence ends up substituting for authorization.

1. LLM reasons and drafts
2. Proposed action
3. Decision model answers narrow typed questions
4. Control layer applies authority, coverage, threshold, tier
5. Execution, or a person

The order is deliberate. Authority is checked first and deterministically, so the model is never consulted on an action the agent may not take. It is consulted only after registration and coverage checks establish that its answer can be relevant, and the decision’s tier sets the threshold that answer must clear.

TypeSafe’s own guidance is to break a judgment into several narrow questions and combine them in your own formula rather than asking one broad question. For an agent proposing a refund: is the amount within its delegated authority? Would an error materially harm the customer? How reversible is the action? The first is a rule, not a model question, and that is the point — anything that can be computed deterministically should be.

Because the final action comes from code, every decision leaves an auditable record: the state supplied, the exact question wording and version, each probability returned, the threshold applied, the rule that fired and the action taken. That is far easier to examine than a paragraph of model reasoning.

Not every decision deserves the same rigor, and treating them alike wastes effort on trivial decisions while under-controlling dangerous ones. Validation depth, threshold strictness and human involvement should scale with three properties of the decision, not of the model:consequence, what an error costs a customer, the firm or a third party; reversibility, whether a wrong action can be caught and undone before harm lands; and autonomy, whether a person sees the output before anything happens or the output triggers action directly. The same model can serve decisions at every level. The tier belongs to the decision contract.

1. Advisory.

   Suggests a queue or a tag for a person to confirm. The human decides every case, and the probability is shown rather than enforced.

2. Routing.

   Routes cases automatically, with easy correction. The human reviews samples and exceptions, and low-probability cases go to a person.

3. Conditional automation.

   Approves or blocks a bounded action within set limits. The threshold is set from error costs on validated data, and the human handles everything below it.

4. Consequential action.

   Informs a consequential, hard-to-reverse action. A human approves before execution; the model informs and never authorizes. Default deny, failing closed on any gap.

The tier names are illustrative. What matters is that a firm fixes the tier when it registers the decision, and that the tier then dictates both the controls and the evidence required. Thresholds come from error costs: if a wrong approval costs nine times a missed one, the threshold sits at 0.9. But that rule has a hidden precondition — it is only valid if the probability is calibrated on the population where it is applied. Applied to an uncalibrated probability it still produces a number, but the error rates it was meant to deliver will not appear. Which is why calibration evidence comes before threshold-setting, and why a vendor-suggested threshold is a default, not a control.

Decision models change where model risk lives rather than removing it. The risk moves out of parsing and prose and into three places: the wording of each decision contract, the meaning of each probability on the firm’s own population, and the boundary between what has been validated and what an agent can reach. The response is architectural and proportionate. Register every decision the model may influence. Tier it by consequence, reversibility and autonomy. Calibrate it against outcomes rather than consensus. Pin its version and control its wording. Wrap it in deterministic checks for authority and coverage, and send whatever falls outside to a person.

The governing principle is simple to state and harder to practice:use AI to provide intelligence, and use governance engineering to decide when that intelligence is allowed to act. Model confidence should never substitute for authorization.

Sources. TypeSafe AI, Introducing System One Models & Jev (15 September 2026), together with its product documentation and homepage; The New Stack; MarkTechPost (19 September 2026); You.com; Pere Pages, Jev, Sorted: What Is Still Just a Claim (16 September 2026); Spring blog, Spring AI and TypeSafe Jev (21 September 2026); Vercel Knowledge Base; Opper gateway documentation; the Awesome Jev community reference; TrueFoundry. All pages accessed 24 September 2026. Full citations in the whitepaper.

Provided for general information; not legal advice. AI Risk Consulting is not a law firm. Jev entered early access in September 2026; product behavior, pricing and version policy may change.