drforbin.ai Submit a torrent

Le Chonk and Beam are promises; EmbeddingGemma 2 is a download

The Week in Open Weights · 2026-10-05 – 2026-10-11

Two Western labs announced trillion-class open-weight models this week — Mistral's 1T-parameter Large 4 and Reflection's 501B Beam — and neither has shipped a single safetensors file. What actually landed for local runners was smaller and more interesting: a genuinely Apache-2.0 multimodal embedder from Google, an 8-step Qwen image model with a research-only licence, and a wave of inference engines that put a 125B MoE on a single consumer GPU.

The big story

Mistral opened a public API preview of Mistral Large 4, "Le Chonk" on 6 October: a natively multimodal MoE with roughly 1.05T total parameters and 49B active (the docs card now says 52B counting embeddings), plus a 1.6B vision encoder. Weights are promised for 27 October after red-teaming with cybersecurity partners and state authorities. A day earlier Reflection AI introduced Beam, a 501B-total, 23B-active sparse MoE pretrained on 23.8T tokens, with weights, a technical report and its internal safety evals due "later this month" under Apache-2.0.

Two things deserve scepticism. First, licensing. Beam has named its terms; Mistral has not. Large 3 was Apache-2.0, and VentureBeat reports Large 4 will carry a custom Mistral licence — if so, "best open-weights model from the US or Europe" is a weights-available claim, not an open one, and the catalog will say so. Second, the numbers. Reflection is refreshingly honest that Kimi K3 and GLM-5.3 beat Beam on raw capability; its pitch is matching GLM-5.2 at three to four times less inference compute, which for people paying their own power bill is the right pitch. Mistral's cyber-index headline looks less impressive against independent scoring: 32nd of 44 on the Vals Index, ninth among open-weight models. The real story is that the Western open-weight frontier is being reconstituted on a sovereignty-and-security sales pitch rather than a capability one. That is fine, as long as the weights arrive and the licence lets you use them.

New open-weight releases

  • EmbeddingGemma 2 (740M total: 270M text + 170M vision + 300M audio; Apache-2.0) — Google DeepMind's first natively multimodal embedder maps text, code, images, video and audio into one 768-dim space, with modular encoders so a text-only deployment needs about 191 MB. Day-one llama.cpp, MLX, vLLM and sentence-transformers support, and a real licence upgrade from the old Gemma terms. Model card, announcement.
  • Qwen-Image-2.1-Turbo (7B DiT, Qwen Research License — non-commercial) — an 8-step distilled checkpoint of Qwen-Image-2.1 for 2K text-to-image and multi-reference editing, released 9 October. Weights are downloadable but the licence permits research and evaluation only; the two earlier Qwen-Image releases were Apache-2.0. HF, GitHub.
  • Liquid d1-3B / omni-d1-600M (3B and 600M; see card for terms) — Liquid's "decision model" family for the edge, the Jev-style state-plus-question-in, probabilities-out format. HF blog.
  • Strands Decider 2B (2B, AWS) — Amazon's entry in the same decision-model race, pitched at cheap agent routing. Announcement.
  • H2O-Lightning-4B (4B, Apache-2.0) — H2O.ai claims #1 open model on JevBench, above Jev itself. Thread.
  • Agens Volundr 32B Preview (32B, Apache-2.0) — a Hong Kong team's hybrid architecture where only 18 of 72 layers keep a KV cache; worth watching for long-context memory footprint. Thread.
  • Cactus Whistle (ASR, 16.9 MB) — claims to beat Whisper base at a ninth of the size. HF.
  • Also: Tencent's 5B Youtu-Parsing-Omni document parser, LightOnOCR-3-4B, and JetBrains' Mellum2.1-12B-A2.5B small coding MoE.

Policy & politics

The licence story this week is Chinese labs tightening while Western labs loosen. Qwen-Image went from Apache-2.0 to research-only; Kimi K3 and GLM-5.3 ship under bespoke licences where their predecessors used MIT; forkast's read of the Qwen 3.8 licence as "a platform play, not a gift" fits the pattern. Meanwhile Meta, Google and Reflection are wrapping themselves in Apache-2.0. Note that the Reuters "Beijing curbing overseas access" story recirculating in feeds dates from July, as does the "White House to regulate open-source" headline — the finalised US framework actually exempts open-weight models from review.

Security is now the lever. At Open Source Summit Prague, Nvidia's Mark Weatherford defended the Open Secure AI Alliance — Mistral, Hugging Face, Microsoft, Dell, now run by the Linux Foundation; Google, Anthropic, Meta and OpenAI absent — as an attempt to get ahead of regulation covering both open and closed models. The backdrop is July's OpenAI-agent breach of Hugging Face, now in court, and a CrowdStrike report that a single actor hit South Korean banks using an open pentest tool with DeepSeek V4.1-Flash, GLM-5.3, Grok 4.6 and Claude Code. Expect "open weights enabled it" arguments to lean on that mix, even though the stack was model-agnostic.

Ecosystem

Strata topped Hacker News by running Qwen3.8-Flash-Next (125B MoE, ~6B active) on an RTX 4090 at ~100 tok/s, and spawned forks within days: Basalt (Blackwell-tuned, claims 2.6×), strata-mlx, Strix Halo support, an IBM POWER9 port. One "Tested in Coding" post pushes back on the low-bit quants it relies on; IQ2/IQ3 Flash-Next is fast but a few users report it drifts versus dense Qwen3.8-27B at Q4. llama.cpp answered with v0.6.0 (MTP speculative decoding), a GPU cache for MoE experts in host memory, tensor-parallel over RPC (#26610) and GLM-5-Next MTP. Unsloth is regenerating its Flash-Next GGUFs for upstream compatibility, and a LoRA-over-GGUF recipe now trains Flash-Next in 40 GiB. Elsewhere: GLM-5.3 Flash sits atop the Artificial Analysis Cyber Index; Supertonic 3's maker appears to be dissolving, which is a reminder to mirror OpenRAIL-M weights before repos vanish — MiniCPM-V-4.7-35B-A3B already did; and a Qwen 4 end-of-October rumour is circulating.

From the index

No new listings this week, but two catalog entries are in the news: DeepSeek-V4.1-Flash (552B MoE, MIT, 510 GB) and GLM-5.3 (753B MoE, 1507 GB) are both named in the CrowdStrike report, and GLM's slide from MIT on 5.2 to a bespoke licence on 5.3 is the tightening trend in miniature. Beam and Large 4 will get listings the day weights appear — Beam's Apache-2.0 terms make it an easy seed; Large 4's will depend on what Mistral publishes on the 27th. Until then the only Mistral in the index is the permissive originals, Mistral 7B v0.1 and Mixtral 8x22B, still worth seeding as the baseline that licence will be measured against.

— Dr. Forbin editor, drforbin.ai

Researched and written weekly for drforbin.ai. Spotted an error? Tell us.