FlexDM and FlexDM-LoRA: Deciding by Reading, Not Generating

The arrival of Jev

On September 15, 2026, TypeSafe AI announced a new class it calls “System One models” and its first model, Jev (announcement). The premise is this. Chat language models have been getting smarter for years, yet they have not automated a matching amount of work. What a model produces is text, so a program has to parse and validate it every time. That adds cost and latency, and a response in the wrong format can break the flow.

The name comes from Daniel Kahneman’s Thinking, Fast and Slow. Human thinking has a fast, intuitive System 1 and a slow, deliberate System 2. If reasoning models that think at length before answering are close to System 2, Jev concentrates on making small decisions quickly, like System 1.

These are the properties TypeSafe has stated for Jev.

• It does not generate text. Every answer comes out of a single parallel computation.
• End-to-end response time is 70 to 500 ms. On System One shaped queries, TypeSafe says this is 40 to 200 times faster than current large models.
• Input costs $0.042 per million tokens, and output is free.
• Every answer comes with a calibrated probability. Format errors, where the answer falls outside the predefined options, cannot happen by construction.
• The training method is described only as RLCD (Reinforcement Learning for Calibrated Decisions): the reward goes to answers with honest probabilities, rather than to human preference (RLHF) or verifiable correctness (RLVR). The data and the architecture have not been disclosed.

Jev is available only through an API, and its weights are not public. An independent evaluation paper (arXiv 2609.37647) measured Jev zero-shot on 37 datasets spanning classification, routing, content moderation and rubric grading. Jev beat Qwen3.8-27B on 27 of the datasets and Gemma-4-E4B on all 37, and its option probabilities were well calibrated. All three models, however, struggled with low-resource languages, noisy labels and rubric grading, and in multi-step game tasks Jev sometimes repeated actions that had no effect.

What a System One model does

A System One model takes a “state” and “typed questions” and returns, for each question, an answer in a fixed format together with probabilities. The state can be anything: a document, a conversation, an email, a list of devices. There are three question types.

For example, a program that routes customer inquiries asks this (some fields are omitted).

{
  "state": "The laptop I ordered still hasn't arrived after a week. I think my card was charged twice, too.",
  "questions": {
    "team": {
      "type": "choice",
      "instructions": "Which team should this inquiry go to?",
      "criteria": {
        "shipping": "Delivery delays, lost packages",
        "billing": "Payments, refunds",
        "tech": "Product faults, how-to questions"
      }
    },
    "urgent": {
      "type": "noul",
      "instructions": "Does this inquiry need an answer today?",
      "criteria": { "true": "Money is at risk or a deadline applies", "false": "Otherwise" }
    }
  }
}

The answer looks like this.

{
  "team":   { "type": "choice", "choice": "billing",
              "probabilities": { "shipping": 0.38, "billing": 0.57, "tech": 0.05 } },
  "urgent": { "type": "noul", "noul": 0.81 }
}

The program does not parse the answer; it branches on it directly. Honest probabilities make it more useful still. Only ambiguous decisions like “billing 0.57” need to go to a person, and high-confidence decisions can be handled automatically. That is why, for a System One model, calibration matters as much as accuracy.

Jev Decision Index

On September 22, a week after Jev was announced, the Jev Decision Index opened. It is an independent leaderboard, unaffiliated with TypeSafe, where open models that behave like Jev compete. The first edition measured 31 open models and Jev 1.13 on the same 132,422 requests. The board then changed quickly: a private test was added on October 6, and since October 20 the ranking has been based on the private test alone.

A model receives a state such as a document, a conversation or a device list, and answers choice and noul questions with a probability for every option. The rules are strict.

• Answer with a single readout, without generating text.
• On one RTX PRO 6000, the median, mean and p80 latency must all be 1,000 ms or less. Anything slower is kept off the board as “no longer Jev-like”.
• Truncating the input, dropping options, benchmark-specific prompts and training on test items are all forbidden.
• The ranking score (Full) is computed from the private test only, not the public one. It combines “same-skill” (60%), which tests the same abilities on new data, and “new-domain” (40%), which covers fields the model has not seen.

Jev itself is on the same board as the baseline. Jev 1.13 has a Full score of 59.22 (same-skill 58.04, new-domain 47.72). Its latency is the round-trip time of API calls made from the lab, so it cannot be compared directly with models measured on a single GPU. On the November 6 board, 62 of 274 entries have a higher Full score than Jev, and 8 of those are inference techniques that leave the weights untouched. That said, Jev was built for real workflows rather than for this board, and new versions keep arriving.

Our two entries

We submitted two entries built on the same base model, google/gemma-4-31B-it.

• FlexDM: an inference technique that changes only how the model reads, without touching the weights
• FlexDM-LoRA: a LoRA adapter trained in FlexDM’s reading layout

When FlexDM was first listed (October 21), it ranked 8th of 224 and 1st among inference techniques. The new-domain score of FlexDM-LoRA ver 2.1 is the 2nd highest on the whole board.

This post does not cover only what worked. It also records what we tried and dropped, and the calls we got wrong at first.

1. Read instead of generating

We do not let the model write an answer. This is the simplest way to make an ordinary language model answer like a System One model. The prompt ends at Answer: (, and at that position we read only the log-probability \(\ell_i\) of the next token being each option letter (A, B, …). The probabilities are a softmax with temperature \(T\).

The temperature depends on the question type. For multiple choice with \(n\) options it is \(T = 10.9757 - 1.7016 \ln n\), and for yes/no it is \(T = 10.732\). We fitted it on half of a calibration set drawn from outside the public test and checked it on the other half. On the held-out half, the expected calibration error (ECE) dropped from 0.123 to 0.030. Temperature does not change the top answer, so accuracy stays the same and only the probabilities become honest.

We dropped the “yes/no bias correction” that was fitted alongside it. The yes/no calibration items came from a single dataset, which made the evidence weak, and it lowered the public sample index from 61.03 to 60.72.

2. Show the questions first

The usual layout gives the context first and the question after it, so the model reads the context without knowing what to look for. FlexDM shows the list of questions in the request once, before the context.

Questions that will be asked about the context:
- Question 1
- Question 2

Context:
<context>

Question: Question 1
Options:
(A) ...
(B) ...
Answer: (

On 2,000 public-sample requests, this layout alone raised the sample index from 55.45 to 61.03, a gain of +5.6. The baseline is the context-first layout, the same one used by the board’s “Decider chat” entry. The questions-first layout itself was already in use by other entries on the board.

3. Read short requests twice

A decoder model can only see earlier tokens. On the first read, therefore, the context tokens cannot see the question and options that come after them. For requests of 3,000 tokens or fewer, FlexDM writes the entire user input once more.

<question list> <context> <question and options>

Read the same input again before answering:
<question list> <context> <question and options>
Answer: (

Every token in the second copy can see the whole first copy. This gives the effect of bidirectional reading without changing the weights.

We had also tested reading twice on our first engine, built on transformers. The sample index rose by +3.44, but the mean latency was 2,414 ms, over the limit. We could only bring it back after moving to vLLM gave us latency headroom (section 4).

Reading twice gained +1.79 (90% interval +0.17 to +3.40). Raising the limit to 6,000 tokens left the score almost unchanged and made it slower, so we stopped at 3,000.

The length limit came out of a failure. At first we read every request twice. In three hours, only 402 of 2,000 requests finished, and one long retrieval request took 1,446 seconds, 28 times slower than reading the same request once. The cause was the prefix cache. The first copy ends with options that differ per question, so the second copy after it has to be computed from scratch for every question. In requests whose question list alone runs to tens of thousands of tokens, we were recomputing tens of thousands of tokens once per question. Short requests are cheap to recompute, hence the length limit.

Reading twice had a side benefit. The share of questions whose top answer changes when the option order is reversed fell from 9.19% to 7.21%. The model is less sensitive to order.

Feeding the input twice is not new either. Other entries on the board were already doing it. FlexDM’s contribution is to set the length limit so that the cache survives on top of the questions-first layout, and to fit the temperature and the serving to it.

4. Serving fast

Our first engine read questions one at a time with transformers. Its sample index was 55.08, but latency was 1,820 ms on average and 3,873 ms at p80, far over the limit, because for a request with dozens of questions it reread the same context for every question. Even when the questions of a request were batched, p80 was 1,190 ms. So we moved to vLLM.

• vLLM 0.30, FP8 quantization (block-wise) at load time, prefix cache. The questions of one request share the cache up to the end of the context.
• A maximum of 131,072 tokens. On the first engine a 34,330-token request hit the 32,768-token limit and was rejected. A rejection scores zero and truncation is forbidden, so we raised the limit. Requests that still exceed it are marked as unprocessable rather than truncated.
• vLLM returns log-probabilities after Gemma’s logit soft cap is applied. These differ from the values of the transformers engine (before the soft cap), and retrieval tasks, which rank by the probability of “yes”, scored 3 to 4.5 points higher. Later, when training the LoRA, we put the loss on these values as well.
• FP8 and bf16 agreed on the top answer 98.98% of the time. When the organizers reran the private test, the reproduction agreement was 100.00% (831 rows).
• The full public suite of 140,178 rows completed without errors. On the lab GPU (a power-limited Max-Q) the median was 146 ms, the mean 403 ms and p80 534 ms; on the board’s official hardware it was 108.6 / 300.9 ms.

5. Another team adopts read-twice

We released the FlexDM code under MIT. One day after it appeared on the board, a LoRA team (KnowLine) submitted three new versions that kept their weights and added only FlexDM’s read-twice. Their PR states “The layout follows FlexDM (MIT)”. Because their setup reads each question separately, 94% of requests were read twice. On the board, the scores of the same weights changed as follows.

The 35B model’s Full score rose from 61.20 to 65.77, moving it from 37th to 21st.

This was an experiment we could not have run ourselves. The weights are identical and only the reading layout changes, so it shows the effect of reading twice on the private test directly. In particular, new-domain rose by +3 to +6.

That was very different from our own measurement. To avoid fitting to the public score, we had built a separate scorer that imitated the private test using datasets from outside the public test. That scorer measured the new-domain effect of reading twice as +0.13. In fact it had already failed validation before that: it barely detected the training effect of another Gemma LoRA whose board score was known (new-domain +8.36 on the board, −0.43 on the scorer). We still thought it was usable for comparing reading layouts on the same weights, but it missed the new-domain effect of reading twice too. After that we trusted the scores of other entries accumulating on the board more than our own scorer.

6. What we tried that did not help

The reversed re-read at the top looked the most promising at first. On the transformers engine, applying it to the least confident 5% gave +1.42. After the move to vLLM it became +0.79, and on top of read-twice and the fitted temperature it turned negative. We had picked the “5%” ratio on the same sample, and we think reading twice had already taken the share that the reversed re-read used to fix.

The noise was also larger than expected. When we reran with only the number of tokens per batch changed, the top answer agreed 99.58% of the time, yet the sample index moved by 0.5. So we treated differences of around 0.5 as noise and adopted only what cleared a threshold fixed in advance. On the untrained model, reading twice had already absorbed most of the order problem. That conclusion is reversed for the LoRA (section 8).

7. FlexDM-LoRA: training in the same reading layout

Principle

The training input is built exactly like the serving input. We keep the questions-first layout, the read-twice rule at 3,000 tokens and reading all questions of a request at once, and only shuffle the option order per question. The loss is also placed on the probability \(p_\theta\) that is read at serving time (with the same temperature).

The teacher \(p_{\text{teacher}}\) is not a different model but the frozen base model itself.

• For most items: the distribution read from base Gemma with FlexDM, averaged over the forward and reversed option orders. This ties the adapter to the base model.
• For calculation and reasoning items (from v2.0): the distribution of answers that base Gemma reached in thinking mode. Serving does not think, so this moves the gain from thinking into a single readout.

The LoRA uses rank 64 on all seven projections, dropout 0, learning rate 1e-4 and one epoch.

Strength

We do not apply the whole learned update. It is added to the weights as \(W' = W + s\,\Delta W\), and the strength \(s\) is chosen by a rule fixed in advance. The first training run showed us why.

• A pilot adapter trained on only 10,000 items gave +0.42 on the public sample.
• Training on all 90,000 items, for about 31 hours, gave −0.66 instead. Some tasks broke: POP909 chord judgement fell from 48.3 to 22.5, phishing email judgement from 94.9 to 74.4, and home device control from 80.0 to 62.5.
• Using the same adapter at strength 0.5 gave +1.29, and the average over five knowledge and reasoning tasks became +3.58. Tasks like device control returned to the FlexDM level, but phishing email judgement stayed low (section 8).

The per-task numbers above were measured on about 40 items per task, so they swing a lot. At strength 0.5, POP909 looked like −9; remeasured on 1,000 items it was +2 to +3. After that we remeasured the important tasks on all of their items.

Roads not taken

• Benchmark training splits: we started a run that added 36,000 more items from the official training splits of the public test (not the test splits) and stopped it midway. Looking at the board, other Gemma LoRAs that leaned on training splits had new-domain scores 2.6 to 3.5 lower than before training. Even in ver 2.1, training splits make up only 1.6% of the data.
• A rushed run: we started the run after ver 1.0 once and stopped it after three minutes. The broad data had too few items per source, and the thinking-mode labels did not exist yet. After that we started a main run only once all data preparation was finished.

Versions

• ver 1.0: human-labelled public data and synthetic tasks whose answers are computed by code.
• ver 2.0: broader coverage.
  • 359 commercially usable sources from tasksource
  • Knowledge and reasoning training splits with thinking-mode labels
  • Documents and questions written by the base model itself
  • Device control problems built from the kit’s simulator rules
  • POP909 chord problems on songs that are not in the public test
• ver 2.1: filled in the formats where ver 2.0 was weak.
  • 744 product reviews and 1,458 phishing and legitimate emails written by the base model itself
  • Causal graph problems and fictional clinical trial problems generated by code
  • “Which API to call next” problems built from the API definitions in API-Bank
  • Two runs on the same data and settings with different seeds, averaged

Every training row was checked against the 140,620 rows of the public test by exact match, 8-grams, long common spans and fuzzy matching of short lines, and overlaps were removed. We did not use the outputs of other entries.

8. What the LoRA taught us

A thinking-mode teacher transfers into a single readout. In a trial run on only the 10,800 thinking-labelled items, accuracy on 800 held-out items rose from 80.6 to 84.8 (+4.1, 90% interval +2.4 to +5.9). Calculation items gained +9.2. This trial run also included the cross-entropy on the correct answer, however, so we could not separate the share due to the thinking labels alone.

Training plants a lean toward “no”. ver 2.0 became much worse at judging phishing emails. Under the selection rule we had fixed in advance, there was no candidate left to choose. Since the other tasks had risen a lot, we submitted it with a single exception, for the phishing floor.

The cause was not obvious. We had never trained on phishing problems. We first suspected the teacher, but averaging the base model over forward and reversed orders did not change the phishing score. When we collected the missed emails, the errors were phishing judged as legitimate. The content looked plausible, but the hosting was suspicious, as when a personal webmail address asks for material through a shared document link. In the end the culprit was the spam and toxicity sources. They contained more “not harmful” answers than the base model would give, and that pushed every yes/no judgement toward “no”. For ver 2.1 we did two things.

• We removed 287 negative-answer rows from those sources to match the ratio to the base model.
• We added phishing and legitimate emails that we wrote ourselves. The legitimate emails contain no shared document links and no personal webmail senders.

On our task set, phishing judgement went from −15.2 to +5.0 relative to FlexDM.

Order averaging becomes useful again on a trained model. “Re-read only questions with confidence below 0.6 with the options reversed, then average” did nothing on untrained FlexDM, but it was a gain on the trained adapter. The stronger the training, the larger the gain. We think training creates habits tied to option position or label, and the reversed order cancels them. Only low-confidence questions are reread, so latency grows only slightly. It has been included since ver 2.0.

The average of two runs is exact when you concatenate ranks. Concatenating two LoRAs (\(\Delta W_k = B_k A_k\), each of rank 64) gives a single rank-128 LoRA whose update is exactly their average.

The adapter can be loaded into vLLM as it is, without merging.

On vLLM, v_proj has to be copied. Ten global-attention layers of Gemma 4 have no v_proj; V is the output of k_proj. In transformers, a LoRA on k_proj changes K and V together, but vLLM keeps a separate LoRA on the V side. In the serving copy we therefore also copy the k_proj LoRA of those layers to v_proj. If this is skipped, part of the learned update silently disappears, with no error.

9. Predicting the board score

The private score only arrives a few days after submission. So we estimated the private score from the public score ahead of time and waited for the result. For ver 2.1 the estimate was 71.60 and the actual score was 71.61.

That is not skill alone, though. Same-skill was off by 0.69 on the low side and new-domain by 0.88 on the high side, and the two cancelled out. Estimating other entries on the board the same way gave values about 2 points higher than the actual scores on average, so some luck went into this exact hit.

10. Next

• An analysis of why questions-first plus read-twice works, looking at where attention goes
• A model that gives the same answer when the option order changes
• Serving that reaches the same score faster

Links

• Code (FlexDM, MIT): github.com/1000luca/FlexDM
• Adapters (CC BY-SA 4.0): ver 2.1, ver 2.0, ver 1.0
• Public test results: FlexDM, FlexDM-LoRA ver 2.1
• Jev: TypeSafe’s announcement, independent evaluation paper (arXiv 2609.37647)
• Leaderboard: multimodalart/jev-decision-index, evaluation kit: apolinario/decision-index