Find the detail.
Fact retrieval at 32K context. At 8K, all 20/20 checks passed.
01 / PROJECT ANNEALMODEL COMPRESSION & RECOVERY
Keep the capability.
Lose the weight.
A 27-billion-parameter model, from roughly 53.8 GB to 9.03 GB. Our latest recovered model averages 97.35% retention of the original model’s scores across seven core benchmarks.
9.03GB27B model package
97.35%Core capability retained
32KContext evaluated
Model files, in decimal GB. Package size is not total running memory.
Original teacher = 100%. Average of each task’s retention, without realtime correction. Retention is separate from answer accuracy.
Compress.Store the existing model’s weights more efficiently.
Recover.Train scales and norm weights to bring capability back.
Compare.Evaluate against the original model under the same protocol.
Recovery leaves the compressed codebook and indices unchanged. A small trainable part brings the model closer to its original behaviour.
Selection set, thinking enabled: divergence falls about 63%. Independent confirmation set: 0.625 to 0.254, down 59%. Lower is closer to the teacher.
LONGER THINKING. MEASURABLE RECOVERY.
About 63% closer on this measure.
Selection-set divergence fell from 0.890 to 0.326. An independent confirmation set fell from 0.625 to 0.254, down 59%. Both measure how closely the model follows the original, separate from task accuracy.
27B release candidate · updated recoveryHow to read these results
Thinking enabled, compared with the full-precision BF16 model. The seven-core mean is 97.35%; the reported average of twelve per-task retention results is approximately 96.00%. Model-only results exclude realtime correction. Retention divides each task’s score by the original model’s score, then averages those ratios. The lowest core-task retention is 93.98% on MATH-500. MMLU declined from 98.09% to 96.18%; the overall core average improved.
The new divergence test uses thinking-on examples up to about 40K tokens, versus 1,024 in the earlier test. Raw divergence values are not directly comparable across those tests; compare the drop within each test.
Recovery still trains scales and norm weights. The trainable share was previously measured at 0.71%, but has not been recalculated for this run. The codebook and indices are confirmed unchanged, at approximately 2.69 bits per weight.
11.69 GB peak for the model plus 32K context on Apple silicon, reproduced in 3 of 3 runs. Acceptance on a real 16 GB machine remains open. Package size is 9.03 GB and is separate from runtime memory. Applying this pipeline to other model architectures remains development work.
Reported compute: about 3.3 A100 GPU-hours (US$5.30) for data capture, compression and recovery training. Setup and data generation are excluded. The full benchmark run cost US$2.27.
Read the detailed briefEvery approach. Its own result.
An earlier baseline, three codebook sizes and two recovery stages. Each was evaluated against the original model. The latest recovery uses much longer thinking-on examples.
Earlier compressed baseline88.93% core retention · 8.49 GB package
Our earlier compressed representation provides a reference point for the codebook experiments. The representation uses 2.52 bits per weight; the lowest core-task retention is 78.26%.
| Benchmark | Retention |
|---|---|
| MBPP · coding | 78.26% |
| HumanEval · coding | 83.12% |
| GSM8K · maths | 94.90% |
| MATH-500 · maths | 81.95% |
| ARC-Challenge · reasoning | 96.60% |
| MMLU · knowledge | 94.90% |
| IFEval · instructions | 92.76% |
Thinking enabled, using the fixed task selections and teacher reference. These are individual configurations, not independent repeat runs across model families. Package size excludes working memory.
Compact codebook · 256 entries80.82% core retention · 7.50 GB package
A smaller dictionary of weight patterns prioritises package size. The representation uses 2.23 bits per weight; the lowest core-task retention is 60.9%.
| Benchmark | Retention |
|---|---|
| MBPP · coding | 76.09% |
| HumanEval · coding | 61.04% |
| GSM8K · maths | 91.84% |
| MATH-500 · maths | 60.90% |
| ARC-Challenge · reasoning | 97.96% |
| MMLU · knowledge | 91.08% |
| IFEval · instructions | 86.84% |
Thinking enabled, using the fixed task selections and teacher reference. These are individual configurations, not independent repeat runs across model families. Package size excludes working memory.
Balanced codebook · 512 entries88.65% core retention · 8.26 GB package
A larger dictionary keeps more weight patterns while staying compact. The representation uses 2.46 bits per weight; the lowest core-task retention is 74.68%.
| Benchmark | Retention |
|---|---|
| MBPP · coding | 75.00% |
| HumanEval · coding | 74.68% |
| GSM8K · maths | 94.90% |
| MATH-500 · maths | 81.20% |
| ARC-Challenge · reasoning | 99.32% |
| MMLU · knowledge | 98.73% |
| IFEval · instructions | 96.71% |
Thinking enabled, using the fixed task selections and teacher reference. These are individual configurations, not independent repeat runs across model families. Package size excludes working memory.
Larger codebook · 1,024 entries90.27% core retention · 9.03 GB package
More representative weight patterns give the strongest core retention of these pre-recovery codebook variants. The representation uses 2.69 bits per weight; the lowest core-task retention is 81.17%.
| Benchmark | Retention |
|---|---|
| MBPP · coding | 81.52% |
| HumanEval · coding | 81.17% |
| GSM8K · maths | 93.88% |
| MATH-500 · maths | 87.22% |
| ARC-Challenge · reasoning | 97.28% |
| MMLU · knowledge | 98.09% |
| IFEval · instructions | 92.76% |
Thinking enabled, using the fixed task selections and teacher reference. These are individual configurations, not independent repeat runs across model families. Package size excludes working memory.
Earlier targeted recovery · previous model92.51% core retention · 9.03 GB package
Earlier recovery trained 0.71% of parameters while keeping the compressed codebook and indices fixed. The representation uses 2.69 bits per weight; the lowest core-task retention is 86.47%.
| Benchmark | Retention |
|---|---|
| MBPP · coding | 91.30% |
| HumanEval · coding | 87.66% |
| GSM8K · maths | 95.92% |
| MATH-500 · maths | 86.47% |
| ARC-Challenge · reasoning | 98.64% |
| MMLU · knowledge | 98.09% |
| IFEval · instructions | 89.47% |
Thinking enabled, using the fixed task selections and teacher reference. These are individual configurations, not independent repeat runs across model families. Package size excludes working memory.
Long-context recovery · current candidate97.35% core retention · 9.03 GB package
Recover capability on longer thinking-on examples by training scales and norm weights. The compressed codebook and indices are confirmed unchanged. The representation uses 2.69 bits per weight; the lowest core-task retention is 93.98%.
| Benchmark | Retention |
|---|---|
| MBPP · coding | 98.91% |
| HumanEval · coding | 96.10% |
| GSM8K · maths | 98.98% |
| MATH-500 · maths | 93.98% |
| ARC-Challenge · reasoning | 98.64% |
| MMLU · knowledge | 96.18% |
| IFEval · instructions | 98.68% |
Thinking enabled, using the fixed task selections and teacher reference. These are individual configurations, not independent repeat runs across model families. Package size excludes working memory.
The three codebook sizes are variants of one method. Recovery is a separate training stage. All results use the same 27B model family; transfer to other architectures is still ahead.
02 / PROJECT rMLXRUST × METAL × APPLE SILICON
Closer to
the metal.
Our native Rust and Metal runtime runs the compressed 27B model on Apple silicon. Peak memory for the model plus 32K context measured 11.69 GB in 3 of 3 runs.
Keep the instructions.
Instruction constraints at 32K context. Longer-context consistency still needs work.
Complete the tool task.
Original model: 45/60. Retention: 97.78%. Task success, not format validity.
One tested attention operation at 32K context. Lower is better. This is not a whole-model speed comparison.
M5 MAX SHORT-CONTEXT GENERATION
Up to 95.1 tokens per second.
In one M5 Max Mac Studio server run, the 27B model decoded 4,361 sampled tokens at 95.1 tok/s with MTP3 drafting and correction enabled. A separate deterministic code request reached 92.1 tok/s and matched its plain-decoding output token for token. These were separate single runs; decode speed excludes prefill.
A 4-bit attention cache uses about one-third the space of the 16-bit cache, with roughly equivalent measured kernel speed.
Measured peak at 32K: 11.69 GB, reproduced across three runs. Final acceptance on a real 16 GB machine remains open.
Runtime measurements & release status
The tested attention operation at 32K fell from 38.96 ms to 2.165 ms, roughly 18× faster. This is a component result, not a whole-model speedup.
Compared with the previous model, 32K retrieval fell from 20/20 to 19/20 and instruction constraints from 9/10 to 7/10. The earlier 60/60 tool-format check was not rerun. The current 44/60 tool-task score is a different measure.
An additional index optimisation failed its quality screen and is disabled. The speed figures above exclude that experimental path.
Native Rust and Metal execution on Apple silicon. Windows and Linux production deployment are not yet accepted. These focused checks do not establish every agent workflow or complete Harness deployment readiness.
03 / RUNTIME RELIABILITYBETTER BEHAVIOUR. MEASURED SEPARATELY.
Help the model
finish the job.
The new model answers more problems correctly before correction is applied. Around it, twelve server-integrated controls handle repetition, unfinished thinking and malformed requests.
164 problems, thinking enabled. Both models run without the correction policy. These gains come from the updated model.
150 problems, thinking enabled. Both models run without the correction policy. These gains come from the updated model.
| Benchmark | Correct answers | Original model | Retention |
|---|---|---|---|
| MBPP | 91/10091.0% correct | 92/100 | 98.91% |
| MATH-500 | 125/15083.3% correct | 133/150 | 93.98% |
| HumanEval | 148/16490.2% correct | 154/164 | 96.10% |
| GSM8K | 97/10097.0% correct | 98/100 | 98.98% |
Retention compares each score with the original full-precision model. It is separate from answer accuracy. The current model’s corrected benchmark suite has not been rerun.
Software checks passed.
All twelve correction components are built into the server.
Recorded generations replayed.
Zero content violations in this replay test.
Control outputs unchanged.
With correction enabled, release-runtime control prompts produced identical outputs.
One real runaway spent 8,192 tokens thinking without finishing. Correction brought it to a right answer. The larger runaway test is still running; one success is not an overall success rate.
Current results & remaining checksStronger model alone · broader live correction testing continues
Full-suite retention with realtime correction is not yet available for the current model. Model-only mean retention is 97.35% across seven core benchmarks and approximately 96.00% across the broader twelve-task set.
HumanEval output-limit cases fell from 27/164 on the previous model to 13/164 on the new model alone. MATH-500 fell from 26/150 to 14/150. Neither comparison includes correction.
HumanEval repeated blocks occurred in 3/164 current-model outputs, versus 1/164 for the full-precision model. This failed the repetition check. Software and replay passes do not establish reliability across every live task.
Previous model: correction-policy comparisonHistorical results · not rerun on the current model
| Benchmark | Model alone | With policy | Retention |
|---|---|---|---|
| MBPP | 84/100 | 95/100 | 103.26% |
| MATH-500 | 115/150 | 134/150 | 100.75% |
| HumanEval | 135/164 | 151/164 | 98.05% |
| GSM8K | 94/100 | 96/100 | 97.96% |
This was a segmented A100 evaluation on the previous recovered model. The original teacher reference was reused. Gains belong to the combined policy; they cannot be added to the current model’s retention.
Previous-model output-limit cases fell from 27/164 to 1/164 on HumanEval and 26/150 to 5/150 on MATH-500 with correction. HumanEval repeated blocks rose from 0/164 to 2/164. The early thinking cutoff makes those completion counts optimistic relative to native enforcement.
Only four of twelve planned tasks completed. Independent repeated live benchmark runs and the current model’s full corrected retention remain ahead.
Keep the response on track.
Server-integrated safeguards:
- Check that tool requests are complete and readable.
- Keep structured responses and code blocks in the expected format.
- Catch repetition and leave room for the final answer.
- Handle responses that stop early or cannot be read.
- Pass tool results back in order and keep the conversation’s context intact.
- Retry recoverable errors within a set limit.
Twelve components integrated. Broader live validation continues.
04 / E-RLMOUR NEXT RESEARCH FRONTIER
Think it through.
Work it out.
A different boundary between model and runtime. Deterministic execution happens inside active reasoning. Returned values carry into the next thought; retained fragments become a complete file.
01 / PHYSICS
One value. Carried forward.
Follow a GPS clock-drift calculation.
02 / CODE
Write. Run. Refine.
Build a file from tested fragments.
Everything in orange is the runtime’s: produced once, carried forward, never re-typed. Scripted demonstrations.
Calculate in the flow.Results enter the active generation.
Carry the result.Reuse a value instead of predicting it again.
A full file. Assembled.The runtime joins the retained fragments. The model never types the file out again.
Working protocol · Capability proof pending. The runtime executes the protocol. The current fine-tune has learned its notation; improved problem-solving and reliable execution selection remain to be demonstrated.
What comes next?
Test execution on problems the model has not seen: choosing when to run code, using the returned values correctly, producing the right answer and stopping cleanly. The protocol is working; a general reasoning improvement has not yet been demonstrated.
The Rust controller operates inside decoding. An isolated worker executes operations and returns values to the continuing thought. These two scripted demonstrations explain the interaction; they are not benchmark results.