Polybee AI Research · Publication 02
F3-Swarm v1.5.2
Small models, verification and local inference under memory constraints.
- 01Qwen2.5-0.5BTriage and first attempt
- 02Qwen2.5-1.5BOn-demand escalation
- 03Qwen2.5-Coder-3BCode repair using counterexamples
Sequential loading: one active model at a time.
Technical preprint without peer review, prepared for submission to arXiv. Code and execution logs are available on GitHub.
01 / Scientific summary
How far can local inference go with limited memory?
F3-Swarm investigates two problems on commodity computers: the cost of moving weights between RAM and storage, and the reliability of small models on operational tasks.
Track A (Labs 01–02) compares streaming matrices from SSD with a three-process pipeline that keeps weights in RAM. Communication uses TCP loopback on the same machine with emulated network latencies.
In Track B (Labs 03–05), verifiers decide when to accept a small model’s output or escalate to another. The evaluation covers 30 Brazilian Portuguese tasks: 12 triage, 10 document extraction and 8 Python function synthesis tasks.
02 / Results
Two experiments, two measures of performance.
Memory · Labs 01–02
Resident weights in RAM versus SSD streaming
The workload uses 24 layers of dense matrices (1.50 GiB in float32), 32 steps and a 600 MiB working-set limit per process. The hardware is an Intel Core i7-11370H running Windows 11.
| Condition | Steps/s | Throughput gain | SSD reads |
|---|---|---|---|
| Unbuffered NVMe | 1.44 | 1× | 33.00 GiB |
| Emulated LAN · 0.4 ms | 19.79 | 13.70× | 0 GiB |
| Emulated Wi-Fi · 4.5 ms | 15.62 | 10.82× | 0 GiB |
| Emulated WAN · 60 ms | 4.84 | 3.35× | 0 GiB |
The study recorded a maximum numerical difference of 0.0 across compared conditions. Steps are operations in the matrix workload; they are not tokens generated by a language model.
SLM cascade · Labs 03–05
Accuracy and runtime on Portuguese tasks
| Policy | Accuracy | Total time |
|---|---|---|
| P1 · 0.5B | 12/30 · 40.0% | 57.76 s |
| P2 · 1.5B | 22/30 · 73.3% | 111.18 s |
| P3 · Initial cascade | 19/30 · 63.3% | 96.21 s |
| P4 · Swarm v1.0 | 23/30 · 76.7% | 84.70 s |
| P5a · Unpatched diagnostic | 27/30 · 90.0% | 119.07 s |
| P5b · Patched version | 29/30 · 96.7% | 121.19 s |
P4 scored 23/30 versus 22/30 for standalone 1.5B (P2) and took 23.8% less time in this test. P5b scored 29/30 (95% Wilson CI: 83.3%–99.4%) but took 121.19 s, 9.0% longer than P2. The 3B specialist was invoked on 3 of the 30 tasks.
P5b incorporated suffix closure and a correction to the expected output for task T26. The set had already informed system tuning; 29/30 describes development performance.
Code generalization audit
| Subset | Cases passed | Tasks passed |
|---|---|---|
| Code recorded verbatim in logs | 27/28 | 6/7 |
| Including the reconstructed T29 function | 31/32 | 7/8 |
The adjusted aggregate of 28/30 (93.3%; 95% Wilson CI: 78.7%–98.2%) combines 11/12 triage and 10/10 extraction results from the original set with 7/8 code tasks in the post-hoc audit. Triage and extraction were not evaluated on new tasks.
03 / Methodology
Verification determines when to escalate.
Calibrated triage
Logit calibration and option-order inversion help detect first-option bias. Escalation can use a short rationale and deterministic output closure.
Evidence-based extraction
Motor B checks whether the extracted span occurs in the source text and deterministically normalizes Brazilian Portuguese monetary values.
Functional checking
Each Python function is checked against four development tests. On failure, AACR repair receives a counterexample containing the input, expected output and actual output.
Sequential loading
GGUF models run locally on CPU through llama-server, one at a time. Tiers 1–2 stay below 1.5 GiB active RAM; the on-demand Tier 3 was reported at 2.1 GB.
04 / Limitations
What the evaluation supports.
- One machine, emulated RTT.
The matrix pipeline does not test full language models distributed across physical computers. Jitter, congestion and real network links are outside the experiment.
- The tracks were not integrated.
The cascade ran the models sequentially on the local host. Integrating the pipeline with multi-host inference remains future work.
- Small sample and development tuning.
Results describe 30 tasks and a post-hoc function audit. Finite tests can accept solutions that fail on other cases.
- Estimated energy; isolation not demonstrated.
Energy values in the manuscript are estimates, not electrical measurements. The prototype Python harness does not provide security isolation for untrusted code.
05 / Materials and reproduction
Manuscript, code and evidence.
Download the PDF · CITATION.cff
The reproduction script checks JSON records and the post-hoc audit. To rerun model inference and benchmark telemetry, consult the repository’s scripts and instructions.
06 / Citation
How to cite this work.
Oliveira, H. (2026). Polybee F3-Swarm v1.5.2: Verifier-Guided Heterogeneous SLM Routing, Symbolic Evidence Normalization, and Controlled Memory-Pipeline Benchmarks on Commodity Edge Hardware. Polybee AI Research Laboratory. TR-2026-F3S-05. DOI 10.5281/zenodo.24095260 .
@techreport{oliveira2026f3swarm,
author = {Henrique Oliveira},
title = {{Polybee F3-Swarm v1.5.2: Verifier-Guided Heterogeneous SLM Routing, Symbolic Evidence Normalization, and Controlled Memory-Pipeline Benchmarks on Commodity Edge Hardware}},
institution = {Polybee AI Research Laboratory},
number = {TR-2026-F3S-05},
year = {2026},
month = nov,
doi = {10.5281/zenodo.24095260},
url = {https://polybee.ai/research/f3-swarm-v1-5-2/},
note = {Technical preprint, version 1.5.2}
}
Download BibTeX