Ending LLM bloat: How to integrate Jev and System One decision models into production architecture

A meaningful share of AI automation cost comes from asking a large language model to make decisions that do not require generated text. Routing a ticket to billing, sales, or support; selecting a tool; or screening a request for risk does not inherently require a paragraph. Software needs a value it can execute, not prose for a person to read.

Jev is TypeSafe's System One decision model: an application sends state and typed questions, and receives a choice, score, or probability instead of generated text. The official documentation defines confidence for Choice and Score and a 0–1 probability for Noul. This does not make LLMs obsolete. It gives decisions and expression separate places in an architecture.

The problem: structured output is not automatically a structured decision

JSON Schema, function calling, and Pydantic can make an LLM response parseable. The model still interprets instructions, generates tokens autoregressively, and makes the application wait for that generation. A valid route: billing field does not establish that the route is correct, that uncertainty is handled, or that the call is economical for the workload.

System 1 and System 2: a useful architecture metaphor

Kahneman's System 1 / System 2 distinction is useful here as a technical metaphor. A System 1 layer makes a fast, narrow judgment: which queue, whether risk exists, which tool to select. A System 2 layer reasons across context, explains alternatives, and writes text. A conventional LLM often serves the latter role; decision models such as Jev fit narrow, measurable questions in the former. This is a division of work, not a hierarchy of intelligence.

User request
     │
     ▼
[Validation + authorization] ── invalid ──► reject / explain
     │
     ▼
[Jev: route, risk, human_needed]  ← one state, parallel questions
     │
     ├─ confidence ≥ threshold ─► deterministic workflow / tool
     ├─ confidence < threshold ─► human-review queue
     └─ prose required ─────────► LLM (draft, summary, explanation)
                                      │
                                      ▼
                              audit log + metrics

Three decision primitives: Choice, Score, and Noul

Jev constrains the shape of a decision before the request is sent. Instead of leaving the answer open-ended, the application states which kind of value it will consume. Independent judgments can be collected as one decision package because multiple questions on the same state are evaluated in parallel.

{
  "state": {"subject": "My card was charged twice", "channel": "web", "account_tier": "enterprise"},
  "questions": {
    "route": {"type": "choice", "instructions": "Route to the accountable team.", "criteria": {"billing": "payment, invoice, or collection", "support": "product issue", "sales": "purchase or quote"}},
    "urgency": {"type": "score", "instructions": "Assess operational urgency.", "criteria": ["low", "normal", "high", "critical"]},
    "human_needed": {"type": "noul", "instructions": "Is human review needed because of financial harm or account security?"}
  }
}

Here, route is not a label for a dashboard: it selects a queue, SLA, and access boundary. urgency can drive prioritization. human_needed can stop an automated reply. Keep questions atomic. ‘Route this safely, urgently, and correctly’ combines unrelated judgments and is difficult to evaluate.

Cost and latency: compare the right unit

A universal benchmark would be misleading because context length, caching, output length, region, and concurrency change the result. TypeSafe advertises Jev at $42 per billion input tokens, or about $0.042 per million input tokens. An LLM decision cost includes more than input pricing: system instructions, generated structured output, possible repair calls, and waiting time. Measure p50/p95 latency, tokens per call, fallback rate, and the cost of incorrect decisions with your own traffic.

The TankDev decision benchmark: a measurement contract

A provider demo cannot prove that an architecture is fast or inexpensive for a production domain. Decision quality depends on the data domain, the definition of options, and the cost of a wrong result. TankDev's recommended benchmark runs one closed evaluation set through three paths: deterministic rules, Jev/System One, and a structured-output LLM. The aim is not the largest headline percentage; it is to show which decisions can safely be automated in which layer.

Labeled evaluation set (for example, 500 anonymized requests)
                  │
                  ├──► Deterministic rule ─────────► result + duration + cost
                  ├──► Jev / System One ───────────► result + distribution + confidence
                  └──► Structured-output LLM ──────► result + output tokens

For every result: correct label • p50/p95 • call cost • fallback • human override
Segment separately: channel • language • account type • novel/rare category

This article makes no numerical benchmark claim; it provides a results template. Before publishing actual results, document the data source, sample size, labeling method, model/question version, date range, and exclusion criteria. Without that, a comparison is a marketing chart rather than engineering evidence.

Hybrid production architecture: decision model, policy, and LLM

Do not attach a decision model directly to an external side effect. Put a policy layer between the decision and the action. The policy engine applies thresholds, user roles, transaction limits, operating hours, retry count, and risk classes. The model may say that a vendor payment is high risk; explicit application policy decides whether money can move.

const decision = await jev.decide(ticket);
const autoRoute = decision.route.confidence >= 0.85;
const needsReview = decision.human_needed.noul >= 0.20;

if (!autoRoute || needsReview) {
  await reviewQueue.enqueue({ ticketId, decision, policyVersion: "2026-09-27" });
  return { status: "pending_review" };
}

await workflow.dispatch({ queue: decision.route.choice, ticketId });
// Call an LLM only when a human-readable draft or explanation is required.

Confidence is not authority: calibration and thresholds

A confidence of 0.85 only approaches ‘85% correct’ when the model is calibrated on your domain. Begin in shadow mode: record recommendations, compare them with an existing rule or human outcome, and inspect error by segment. Language, customer tier, channel, product family, and newly introduced categories need separate measurement.

• Log a safe state reference, model version, question/policy version, result, distribution, threshold, and final action for every decision.
• Choose thresholds from false-positive and false-negative cost, not headline accuracy.
• Do not turn low confidence into an automatic rejection; design review, an alternate path, or a request for richer context.
• Compare old and new versions under controlled traffic whenever the model or question text changes.

What Jev does not do

Jev does not write text. It is therefore not the right tool for explanations, long summaries, code, or multi-step research. A fixed output space prevents an invented schema value, but it does not prevent a high-confidence selection of the wrong defined option. ‘Zero hallucinations’ must not be interpreted as correct business decisions. If a deterministic rule already exists—such as amount > 100000—use ordinary code rather than a model call.

Production release checklist

• Write the business goal, allowed actions, and reversibility of each decision.
• Describe every Choice option and Score level with observable criteria.
• Prepare an evaluation set from labeled or human-decided examples; do not reuse training examples.
• Specify a fallback for low confidence, timeout, and service failure.
• Monitor queue delay, model latency, confidence distribution, override rate, error rate, and call cost.
• Apply dual approval, limits, idempotency, and audit logs to high-impact actions.

TankDev treats AI automation as a system of APIs, data, permissions, business rules, queues, and observability—not merely a model call. Read our guide to AI automation with LLMs, APIs, and workflows and the TAARM reliability model. Share the decision points in your automation, and we can design which layer should be code, a decision model, or an LLM.