Release v5.11.0

Release v5.11.0 huggingface / transformers Public Notifications You must be signed in to change notification settings Fork 34.7k Star 167k Release v5.11.0 vasqu released this 10 Jun 16:32 · 1039 commits to main since this release v5.11.0 e7b5b96 Release v5.11.0 New Model additions DiffusionGemma DiffusionGemma is engineered to reduce the sequential bottlenecks of standard causal language models by employing an encoder-decoder architecture specifically optimized for inference speed. During inference, DiffusionGemma leverages multi-canvas sampling, where rather than generating one token at a time, the model iteratively denoises a full block of tokens using a diffusion sampler.

This ModelRelease is relevant to the technology intelligence record because it involves GitHub, DeepSeek, Meta, Cohere. The source article should remain the factual reference for follow-up coverage.

• huggingface / transformers Public Notifications You must be signed in to change notification settings Fork 34.7k Star 167k Release v5.11.0 vasqu released this 10 Jun 16:32 · 1039 commits to main since this release v5.11.0 e7b5b96 Release v5.11.0 New Model additions DiffusionGemma DiffusionGemma is engineered to reduce the sequential bottlenecks of standard causal language models by employing an encoder-decoder architecture specifically optimized for inference speed.
• During inference, DiffusionGemma leverages multi-canvas sampling, where rather than generating one token at a time, the model iteratively denoises a full block of tokens using a diffusion sampler.
• This block-autoregressive approach facilitates text generation at higher speeds compared to traditional sequential generation methods.
• Links: Documentation GPU go brr ( #46540 ) by @gante in #46540 DeepSeek-V3.2 DeepSeek-V3.2-Exp is an experimental model from DeepSeek-AI that introduces DeepSeek Sparse Attention (DSA), a trainable, fine-grained sparse attention mechanism designed to improve training and inference efficiency in long-context scenarios.
• Built on top of DeepSeek-V3.1-Terminus with a 685B-parameter Mixture-of-Experts backbone, it reduces the quadratic cost of attention over long sequences by attending only to a selected subset of past tokens while maintaining virtually identical benchmark performance.
• The work was extended in DeepSeek-V3.2 which pairs DSA with scalable reinforcement learning and achieves gold-medal level results on competition math and competitive programming benchmarks.