Release: v5.16.0

Release: v5.16.0 huggingface / transformers Public Notifications You must be signed in to change notification settings Fork 34.7k Star 167k Release: v5.16.0 Cyrilvallez released this 26 Aug 12:35 · 344 commits to main since this release v5.16.0 93d1bcf This commit was signed with the committer’s verified signature . Cyrilvallez Cyril Vallez SSH Key Fingerprint: O743lyKbz6cYCDq5uTRdzyLHAo4JjdtgAF9Q3yVhAQA Verified Learn about vigilant mode .

This ModelRelease is relevant to the technology intelligence record because it involves GitHub, DeepSeek, Cohere, Mistral AI. The source article should remain the factual reference for follow-up coverage.

• huggingface / transformers Public Notifications You must be signed in to change notification settings Fork 34.7k Star 167k Release: v5.16.0 Cyrilvallez released this 26 Aug 12:35 · 344 commits to main since this release v5.16.0 93d1bcf This commit was signed with the committer’s verified signature .
• Cyrilvallez Cyril Vallez SSH Key Fingerprint: O743lyKbz6cYCDq5uTRdzyLHAo4JjdtgAF9Q3yVhAQA Verified Learn about vigilant mode .
• Release v5.16.0 New Model additions Qwen4-Exp Qwen4-Exp builds on Qwen3.5's hybrid text and multimodal architecture with three key components: GatedResidual (GR), Qwen Sparse Attention (QSA), and Per-Layer Embedding (PLE).
• GR is a Qwen-developed residual architecture that combines Hyper-Connection with GatedNorm.
• It mixes multiple residual streams with fine-grained elementwise gating before each attention and Mixture-of-Experts (MoE) block, then controls how much of the block output is injected back into each stream.
• QSA uses multiple query heads to score compressed key blocks, selects the most relevant contiguous token blocks, and keeps the incomplete trailing block uncompressed.