On 6 October 2026 Mistral put a trillion-parameter model on the table and promised the weights by the end of the month (Mistral). Three days earlier Aleph Alpha shipped Kolibri-1 under Apache 2.0 with full weights on Hugging Face (Aleph Alpha). Open-weight models that run on-premises, with no cloud account behind them, are now close to the frontier.
That changes what an operator can plan for. A hosted model can be withdrawn by a provider, a regulator or a letter you never see. A model whose weights sit on infrastructure you control cannot. But open does not mean runnable. Before any of these models helps a grid, a ground station or a defence program, someone has to do the memory math.
What happened in October 2026?
Three releases in four days moved the ceiling on what can run with no cloud. As of 26 October 2026, this is where they stand.
- Mistral Large 4 was announced on 6 October as a public API preview. Mistral describes a natively multimodal mixture-of-experts model with 1 trillion parameters and 52 billion active (Mistral). Its model page lists 1.05 trillion total parameters and a 1.6 billion parameter vision encoder (Mistral Docs). The Hugging Face placeholder is more precise: 49 billion active parameters per token, 52 billion counting embeddings and output layers, with the planned repository named Mistral-Large-4-1T-A52B (Hugging Face).
- The Large 4 weights have not landed yet. On 26 October the Hugging Face page showed a countdown with 31 October 2026 as the current estimate. No licence has been announced. Mistral’s EU AI Act documentation lists only proprietary and bespoke terms for now and says it “will be updated in case of open weights release” (Mistral legal). Its predecessor, Mistral Large 3, is Apache 2.0 (Hugging Face).
- Kolibri from Aleph Alpha, published on Hugging Face as Kolibri-1, arrived on 3 October: 78.1 billion total parameters, 3.46 billion active per token, contexts of up to 1 million tokens, Apache 2.0, built and trained in Germany and Finland (Aleph Alpha).
- EmbeddingGemma 2 from Google arrived on 6 October: a 740 million parameter multimodal embedding model that runs in about 191 MB of active RAM for text-only weights and about 567 MB for the full model on a phone (Google Developers).
The small end moved too. NVIDIA’s Nemotron 3.5 Bolt E4B, released on 23 October, is designed to pack conversational agent intelligence into as little as about 4 GB (NVIDIA).
Why are open weights a strategic reserve?
An open-weight model is a model whose trained parameters are published for anyone to download and run, under a licence that sets what they may do with it. Once the files are on your infrastructure, nobody upstream can switch them off.
That is what a strategic reserve is: capability you still hold when supply is cut. In June 2026 a single export control letter took two frontier models offline for every user worldwide for nearly three weeks, as we covered in your AI has an off switch. Outages, account suspensions and price changes do the same thing more often and with less notice.
Mistral makes the same argument for its own model. Its launch post says that “losing access to a capability mid-incident can itself become a critical security risk”, and pitches open weights and self-deployment as the answer (Mistral).
A reserve only counts if it is usable on the day you need it. Fuel in a tank you cannot pump is not a reserve. Weights you cannot fit in memory, have not tested on your workload or are not licensed to use are not a reserve either. The rest of this post is about turning a download into a reserve. If you are new to the wider idea, start with what sovereign AI means.
How much memory does an open-weight model need?
Weight memory is total parameters times bytes per parameter. Add the KV cache for your context and concurrency, then a margin for the engine. That is the whole formula.
weights (GB) = total parameters (billions) x bytes per parameter
bytes/parameter = 2 at FP16 or BF16, 1 at FP8, about 0.5 at 4-bit
KV per token = 2 x layers x kv_heads x head_dim x bytes per element
KV total = KV per token x context tokens x concurrent sequences
total = weights + KV total + engine margin (roughly 10 to 20%)
Applied to real models, weights only, in decimal gigabytes:
| Model | Total params | Active per token | Licence | FP16 | FP8 | 4-bit |
|---|---|---|---|---|---|---|
| Nanbeige4.3-33B (dense) | 33.1B | 33.1B | Apache 2.0 | 66 GB | 33 GB | 17 GB |
| Kolibri-1 | 78.1B | 3.46B | Apache 2.0 | 156 GB | 78 GB | 39 GB |
| gpt-oss-120b | 116.8B | 5.1B | Apache 2.0 | 234 GB | 117 GB | 58 GB |
| Mistral Large 3 | 675B | 41B | Apache 2.0 | 1,350 GB | 675 GB | 338 GB |
| Mistral Large 4 | 1.05T | 49B | Not announced | 2,100 GB | 1,050 GB | 525 GB |
Parameter counts come from each model’s Hugging Face metadata or model card (Nanbeige, Kolibri-1, gpt-oss-120b, Mistral Large 3) and, for Large 4, from Mistral’s own pages. The memory columns are our arithmetic.
Real files land close to the formula. Kolibri-1 ships as an FP8 checkpoint of 78.8 GB on Hugging Face, against 78.1 GB predicted. The gpt-oss-120b checkpoint is 65.2 GB rather than 58 GB, because only its expert weights use the 4-bit MXFP4 format and the attention layers, router and embeddings stay at higher precision (gpt-oss-120b config). Treat the 4-bit column as a floor, and read the actual file sizes before you buy. Our GPU memory planning guide covers the engine margin in detail.
Why does a mixture-of-experts model need so much memory?
Active parameters set your speed. Total parameters set your bill. A mixture-of-experts model routes each token through a few experts, but the router can pick any expert on the next token, so every expert has to be resident.
Kolibri-1 makes the gap concrete. It has 384 experts in each layer and uses 6 per token (Aleph Alpha). Each token touches about 3.46 billion parameters. Memory still has to hold all 78.1 billion, more than 22 times as many.
Mistral Large 4 is the same story at a larger scale. A token uses about 49 billion parameters, roughly 5 percent of the model, yet the 4-bit weights alone come to about 525 GB.
Speed follows the active count because generating a token is usually limited by how many bytes the processor must read from memory for that token. At 4-bit, Large 4 reads roughly 25 GB of weights per token. A dense model of the same total size would read about twenty times more. That is why a trillion-parameter mixture of experts can answer at a usable pace on a machine that could never run a trillion-parameter dense model.
So the two numbers answer two different questions:
- Can I run it at all? Look at total parameters times bytes per parameter.
- How fast will it answer? Look at active parameters times bytes per parameter, against your memory bandwidth.
Mix them up and you buy a machine that cannot load the model, or one far bigger than the speed you need.
How big does the KV cache get?
The KV cache grows with every token of context and every user, and on long contexts it can outweigh the weights. Attention design decides how fast it grows.
Three published architectures show the range. These figures use each model’s configuration file on Hugging Face and the formula above.
| Model | Attention layout | KV per token | At 128k tokens | At full context |
|---|---|---|---|---|
| Nanbeige4.3-33B, BF16 cache | 52 full-attention layers, 8 KV heads | 213 KB | 27.9 GB | 111.7 GB at 524,288 tokens |
| gpt-oss-120b, BF16 cache | 18 full plus 18 sliding-window (128) layers | 36.9 KB | 4.8 GB | 4.8 GB at 131,072 tokens |
| Kolibri-1, FP8 cache | 10 full plus 40 sliding-window (513) layers | 10.2 KB | 1.3 GB | 10.7 GB at 1,048,576 tokens |
The sliding-window layers add a small fixed amount per sequence (about 21 MB for Kolibri-1 and under 5 MB for gpt-oss-120b) on engines that cap them at the window. Kolibri-1’s figures assume the FP8 cache that Aleph Alpha uses in its own serving command.
The dense model is the warning. At its full 524,288-token context, one Nanbeige4.3-33B sequence needs about 112 GB of cache, against 66 GB for its BF16 weights. Kolibri-1 needs under 11 GB for a million tokens because only one layer in five keeps the full context.
Aleph Alpha’s own numbers line up with this. It says the 78B model handles 18 concurrent 256k-token requests on two H100s (Aleph Alpha). By the formula that is about 49 GB of cache plus a 78.8 GB checkpoint, inside 160 GB of VRAM. When the budget breaks, it breaks here first, which is why our vLLM out-of-memory guide starts with context length and concurrency.
What does tiering across VRAM and system RAM change?
Memory tiering keeps the hottest parts of a model in VRAM and places the rest in system RAM, so a model larger than the GPU still runs. Mixture-of-experts models are well suited to it, because each token needs only a few experts.
The pattern is now common in open tooling. llama.cpp, for example, has options to keep mixture-of-experts weights in CPU memory while the rest of the model stays on the GPU (llama.cpp). The usual split looks like this:
- VRAM: attention layers, shared experts, embeddings, the KV cache, and the experts the router picks most often.
- System RAM: the long tail of experts, fetched or computed on the host when a token needs them.
The trade is speed for fit. Every time a token needs an expert held in system RAM, the data crosses the PCIe bus or runs on the CPU, and that is slower than VRAM. How much slower depends on how evenly the router spreads its work, so measure it on your own prompts.
Take Large 4 at 4-bit. An eight-GPU node of 80 GB cards has 640 GB of VRAM, enough for the 525 GB of weights with little left for cache. A smaller GPU box with a large bank of system RAM can hold the same weights across both tiers. It will be slower per token, but it runs, and on an isolated site running is the requirement. NVIDIA’s 748 GB deskside DGX Station for Windows, covered in the data centre just moved under the desk, shows where the top end is heading.
At the very small end the same rule holds. A 4-bit, 4 billion parameter model flew on a satellite in April 2026 because it fitted the memory on board, as our post on AI in space explains. Quantisation and tiering decide what fits. They do not decide whether it is good enough.
What does quantisation cost in quality?
Quantisation shrinks the bill, but its quality cost depends on the model, the method and your workload, so it must be measured, not assumed. The good releases tell you how their published numbers were produced.
- OpenAI says all gpt-oss evaluations were run with the same MXFP4 quantisation it ships (gpt-oss-120b).
- NVIDIA quantised Bolt E4B to 4-bit Q4_K_M with calibration on about 1 million tokens of its own reinforcement learning rollouts, and reports benchmark scores on that 4-bit build (NVIDIA).
- Google trained EmbeddingGemma 2 with quantisation in the loop, compressing weights to INT4 and INT8 (Google Developers).
- Mistral publishes Large 3 in FP8, NVFP4 and BF16 variants, and says the NVFP4 build runs on a single node of H100s or A100s (Hugging Face).
A vendor’s benchmark tells you the model can do the task in general. It does not tell you how a community 4-bit build handles your maintenance logs, your procedures or your language mix. Run your own evaluation at the precision you will deploy.
How do you qualify an open model for a critical system?
Qualifying an open model means proving, before it goes live, that you may use it, know where it came from, have tested it on your work at the precision you will run, and can change or roll it back with a record of every version. Six checks cover it.
| Check | What to confirm | What to keep on file |
|---|---|---|
| 1. Licence scope | The licence covers your use, sector and redistribution. Apache 2.0, a custom licence and “not yet announced” are three different answers. | The licence text and version, as downloaded |
| 2. Provenance | Who trained it, where, on what data, and from which published files. | Publisher, repository, commit and SHA-256 of every weight file |
| 3. Evaluation on your workload | It does your tasks well enough, including saying “I don’t know” when the answer is not there. | Your test set, prompts, scores and pass threshold |
| 4. Quantisation impact | Quality holds at the precision and quantisation method you will deploy. | Side-by-side scores at full and deployed precision |
| 5. Update and rollback | You can move to a new version, test it, and return to the last good one without a network. | The approved version list and a tested rollback |
| 6. Record of what ran | For every output, you can show which model, version and settings produced it. | A durable log that survives a lost link |
The licence check is not academic this month. Kolibri-1 and gpt-oss-120b are Apache 2.0. NVIDIA’s Bolt E4B uses the OpenMDW 1.1 licence (Hugging Face). Large 4 has no announced licence as of 26 October, so nobody can yet say what using its weights will allow.
The update check matters for the same model. Mistral says its reinforcement learning run “is still in flight” and that it expects “large and rapid improvements in the weeks and months to come” (Mistral). Fast improvement is good news. It also means two sites can run two different models under the same name unless you pin and record the version.
Kolibri-1 is a useful reference for the evaluation check. Aleph Alpha trained it to abstain when the context does not support an answer, and reports abstaining instead of answering wrong on 44 percent of AA-Omniscience items, up from 15 percent for its predecessor (Aleph Alpha). That is the behaviour to test for in a control room. A wrong answer delivered with confidence is worse than no answer.
What should operators do before the Large 4 weights land?
Do the planning now so the download day is a test, not a scramble.
- Size for total parameters. Budget about 525 GB for Large 4 weights at 4-bit, about 1,050 GB at FP8, then add the cache.
- Size the cache from real use. Take your longest real context and your peak number of users, and run the KV formula with the published configuration once it appears.
- Decide your tiers. Work out what must stay in VRAM and how much system RAM can hold the rest, then measure the speed you get.
- Wait for the licence. Do not plan production use until the terms are published and read.
- Build your evaluation set first. A hundred real tasks from your own work, scored by your own people, will tell you more than any leaderboard.
- Pin and record versions. Hash the files on arrival, approve a version, and log which one ran.
- Keep a smaller qualified fallback. A model like Kolibri-1 fits in a fraction of the memory and can carry the load while a larger one is tested.
Where Sector88 fits
Sector88 is one system to deploy, run and govern AI at every site, connected or completely cut off. It tiers models across VRAM and system RAM so larger models fit constrained machines, keeps serving when the network is gone or untrusted, and keeps a durable record of what ran. That makes an open-weight model something an operator can hold in reserve and prove. More in how our technology works.
A model you cannot fit, test or trace is a download. A model you can is a reserve.