We ran a simple experiment. Take a base language model. Inject facts into its weights, one batch at a time. See how many it can absorb before it breaks.
LoRA broke at 50.
Our method -- Direct Neural Programming -- kept going. Through 5,000. Through 10,000. Through 21,000. On a 20-billion parameter model, through 26,000. The models never collapsed. General capability never degraded. The standard benchmark used to measure model intelligence -- MMLU -- came back essentially unchanged at both scales.
This is not a marginal improvement. This is a different category of result.
The LoRA Collapse
We started on Pythia 1.4B, a small open-source model from EleutherAI. We used the same fact corpus for both methods: CounterFact, the standard benchmark for knowledge editing.
LoRA lasted 50 facts. By that point, general reasoning capability had dropped from 60% to 10%. Perplexity -- a measure of how confused the model is -- exploded from a healthy baseline to 459.6. The model was no longer a functional language model. It was producing incoherent text. Our safety systems triggered an automatic shutdown.
50 facts. That was the ceiling.
DNP on the Same Model
Starting from the same vanilla Pythia 1.4B, DNP loaded 21,015 facts from the full CounterFact dataset. The model maintained coherent language generation throughout. General capability held steady. Perplexity stayed bounded.
The model could not retrieve all 21,015 facts simultaneously -- it settled into a sliding window where roughly 400 facts were retrievable at any given time, with older facts cycling out as newer ones entered. But the model itself never broke. It never stopped being a functional language model.
That is a capacity difference of over 400x before the model even showed stress.
The Novel Facts Experiment
CounterFact is a useful benchmark, but it has a weakness: it measures the model's ability to replace existing knowledge with new associations. A skeptic could argue that DNP is just reactivating latent knowledge the model already had.
So we ran a cleaner test. We compiled 5,000 facts from after the model's training cutoff -- events, discoveries, and information the vanilla model genuinely did not know. We verified this by testing the vanilla model on every single fact before injection. Zero percent recall. The model had never seen this information.
After DNP injection:
- Direct recall: 7.6%
- Paraphrase recall: 6.8%
- General capability: improved from 60% to 70%
- Perplexity: unchanged at 13.3
The model learned things it had never been trained on. It did not lose anything measurable in the process.
Scaling to 20 Billion Parameters
A 1.4B model is small. The question was whether DNP would work at scale, or whether the results were a quirk of a tiny architecture.
We ran the same experiment on GPT-NeoX-20B -- a model roughly 14 times larger. The results were not just consistent. They were better.
DNP loaded 26,081 facts into the 20B model. The retrievable window scaled to approximately 5,000-6,000 facts -- roughly 15 times the 1.4B's capacity, tracking roughly linearly with parameter count. General capability held at 100% through the vast majority of the training run.
The 20B model absorbed knowledge at scale without breaking.
The MMLU Test
General capability checks during training are useful, but they are internal. For an independent measure, we ran MMLU -- the Massive Multitask Language Understanding benchmark -- on both the vanilla and DNP versions of each model. MMLU tests 14,042 multiple-choice questions across 57 academic subjects. It is one of the most widely used benchmarks for measuring what a model knows.
| Model | Vanilla MMLU | After DNP | Delta |
|---|---|---|---|
| Pythia 1.4B | 24.3% | 25.3% | +1.0% |
| GPT-NeoX 20B | 24.7% | 24.6% | -0.1% |
At both scales, MMLU is effectively unchanged. The models gained thousands of retrievable facts and lost nothing measurable on the standard intelligence benchmark.
DNP adds knowledge without measurable cost to general capability. The MMLU results confirm this at two different model scales.
Passidation
We coined the term passidation -- the controlled reapplication of previously injected knowledge to reinforce weight-level retention. Think of it as a consolidation pass, not a training epoch. DNP is not fine-tuning. The terminology should reflect that.
After a second pass over the 5,000 novel facts on the 1.4B model, direct recall improved from 7.6% to 12.4%. Paraphrase recall improved from 6.8% to 10.1%. The capacity ceiling did not break, but retention within that ceiling strengthened.
Passidation is still early. The results suggest that repeated passes can reinforce what has already been injected, but they do not overcome the fundamental capacity limit of the model. That limit appears to scale with parameter count.
What About LoRA at 20B?
We have not yet run a matched LoRA experiment on the 20B model. The 1.4B LoRA collapse strongly motivates the comparison, and we expect degradation, but until the experiment is run, the 20B LoRA result remains a hypothesis. We will update this page when we have the data.
What we can say with confidence: DNP loaded 26,081 facts into a 20B model with zero MMLU degradation. Whatever LoRA's ceiling turns out to be at 20B scale, DNP's floor is already established.
What This Means
The conventional assumption in the AI industry is that modifying what a model knows requires significant compute -- retraining, fine-tuning, or adapter methods that risk degrading general capability. The more you specialize, the more you lose. That tradeoff is accepted as fundamental.
These results suggest it is not fundamental. It is a limitation of the method, not the model.
DNP operates at a different level of the architecture. It writes directly to the layers that store factual associations without disturbing the layers that handle reasoning, language generation, and general knowledge. The result is knowledge addition at effectively zero cost to the model's existing capabilities.
If that holds at larger scales -- and the 1.4B-to-20B progression suggests it does -- then the economics of model customization change significantly. Knowledge updates, domain specialization, factual corrections, and behavioral modifications could become lightweight operations rather than expensive retraining jobs.
The technology behind DNP is proprietary. The method is not published. The results are.
jBlaze is the DNP compiler built by Apollo Raines at SAIQL. Released models are available at huggingface.co/ApolloRaines.