When should you escalate to a larger model? The F3-Swarm cascade

1. The decision a cascade must make

Consider administrative triage, extraction of a value in reais, and a Python function. All three can begin with the same model, but a quality signal for one does not automatically transfer to the others. Triage calls for choice stability; extraction calls for tracing the value back to text; code calls for behavior on specified inputs. A cascade saves work only when it knows what it can accept.

F3-Swarm’s Track B tests this idea on 30 Brazilian Portuguese tasks: 12 triage, 10 document extraction and 8 function synthesis tasks. Qwen2.5 0.5B, 1.5B and Coder 3B models ran on CPU with sequential loading. The limit below 1.5 GiB concerns the first two tiers; the specialist was reported at 2.1 GB. This is neither the footprint of three concurrent models nor an end-to-end memory guarantee.

2. Confidence can follow option position

With two available answers, a system may give weight to option position. A simple test reverses the order, remaps the labels and checks whether the semantic choice remains stable. Comparing only the letters is insufficient: if the correct choice moves from A to B, the stable decision must change letters too. An inconsistency is a reason to seek more evidence.

In the prototype, logit calibration and the inversion check help with triage. Some escalated cases use a short rationale and controlled output closure. These measures serve different purposes: remapping options tests sensitivity to order; constraining and closing output helps obtain a parseable response. Neither guarantees that the final classification matches the real case.

3. Extracting a span and interpreting a value are different tasks

A document may contain a subtotal, tax and final amount in one paragraph. Asking the model for a valid number is insufficient: all three can pass a format check. Require the source span and a rule for identifying the field. In F3-Swarm, Motor B checks that textual evidence occurs in the source and deterministically normalizes Brazilian Portuguese currency.

For example, “R$ 1.234,56” can become 123456 centavos as an integer. This is illustrative: storing centavos avoids confusing thousands and decimal separators and preserves a unit for arithmetic. Evidence confirms the value occurs in the source, but the system still needs to verify that it belongs to the requested field. Duplicate values or ambiguous context may require review.

4. Failure should inform the next attempt

For function synthesis, the prototype checks four development examples. On failure, repair receives a counterexample: input, expected output and actual output. This identifies behavior that needs to change. Sending only “try again” wastes a chance to explain the failure, while sending the entire evaluation suite can compromise the independence of the measurement.

5. What a wrong acceptance decision costs

A verifier can make two operational errors: accept an incorrect output or escalate an output that was already correct. The first circulates a bad result. The second adds cost and latency and can even replace a good solution with a bad one. Measuring only the share of accepted outputs does not distinguish these cases.

Imagine 100 extraction requests: the first tier accepts 80 and four are wrong. In this illustrative scenario, initial coverage is 80%; error among accepted outputs is 5%. Changing the threshold can reduce those four errors and increase the review queue. The choice depends on error consequences, review capacity and response time. The study does not set a universal threshold for that tradeoff.

6. The most accurate policy can also take longer

P4 got 23 answers right, one more than P2, and took 84.70 s versus 111.18 s: a 23.8% reduction in reported total time. P5b reached 29/30 in 121.19 s, about 9.0% longer than P2. Tier 3 was invoked for three tasks. These are results on the development set, not proof of a statistical advantage for every operational workflow.

P5 consolidation reused records of already resolved tasks and added runs on selected cases. P5a preserved the earlier diagnostic; P5b adjusted the output budget and closure, corrected T26’s expected outputs and included an algorithm hint in that task’s repair prompt. The gain between variants does not isolate each change’s contribution. The generalization essay in this series separately examines code produced before the hint.

7. How to find which component improved the policy

Comparing two table rows starts the investigation. To attribute an improvement to option inversion, for example, evaluate the same model on the same tasks with and without inversion, holding the rest fixed. To assess normalization, record how many outputs already identified the right evidence and how many changed after the deterministic rule. To assess the specialist, compare solutions before and after repair without changing the oracle at the same time.

Paired error comparison also helps. Systems with 22 and 23 correct answers may miss nearly the same tasks or fail on very different groups. A per-task matrix shows which cases a policy fixed and which regressed. On a small set, this inspection informs the next experiment better than assigning general meaning to a one-answer difference.

8. Count waiting, reviews and errors too

In a cascade, an escalated request pays for the first model, the verifier and the next stage. It also incurs process switching, weight loading and possible retries. The share taking the expensive path can change with traffic. A global mean hides whether the typical user receives a quick response or a class of cases gets stuck in repeated attempts.

To assess costs, record stage timing, escalation rates by task type, p95 and frequency of human review. Account for the cost of a wrong acceptance and unnecessary escalation. Energy reported in the preprint was estimated from assumed CPU power, without direct electrical measurement; do not turn it into a promise of deployment savings.

9. A contract for each output

For triage, keep the category, option set, decisions after remapping and reason for any abstention. For extraction, keep the field, normalized value, unit and source span. For code, keep the selected function, suite version and repair counterexamples. These records let you compare policies without treating fluent text as evidence of correctness.

Also define what happens when every tier fails. A workflow can return “insufficient evidence,” request a clearer document or send the case for review. Include those outcomes in evaluation: completion counts should not hide delays, abandonment or referrals. A pilot is more useful when it reveals where a policy should abstain than when it forces an output for every input.

10. How to apply the idea to your own workflow

Start with a task whose evidence is explicit. Define acceptance, failure and abstention before choosing models. Collect representative cases with reviewed expected outcomes, separate development from evaluation, and compare against a deterministic rule and a fixed model. Use the same inputs, criteria and execution limits for comparisons.

Then log the reason for routing: unstable option, missing evidence, failed test or timeout. Observe error rates among accepted responses, not just the fraction handled by the first tier. Reserve human review for higher-impact decisions and repeat evaluation when document formats or task mix change. The cascade can then be tuned using signals from the actual workflow rather than relying on a lab aggregate.