• TRM$ 3.461,92 +1,37 % +$ 46,82TRM oficial (Superintendencia Financiera), vigente del 21 nov al 23 nov
  • Euro$ 3.804,64 +1,37 % +$ 51,45Referencia: TRM cruzada con la tasa del Banco Central Europeo del 20 nov; no es la cotización de mercado
  • Peso mexicano$ 181,56 +1,37 % +$ 2,45Referencia: TRM cruzada con la tasa del Banco Central Europeo del 20 nov; no es la cotización de mercado
← Volver a la comunidad
Tecnología

Why are diffusion LLMs like Inception's Mercury replacing sequential token decoding to top 1,000 tokens per second?

Discrete diffusion large language models led by Inception Labs' Mercury architecture and open-weight LLaDA are shattering the sequential memory-bandwidth wall of autoregressive Transformers, refining entire blocks of tokens in parallel across iterative denoising passes to exceed one thousand tokens per second on standard enterprise GPUs.

When we profile traditional autoregressive models like GPT and Claude during inference, text generation is fundamentally memory-bound: because token N cannot be computed until token N-1 is finalized, GPU compute units sit idle waiting to read massive model weights from High Bandwidth Memory for every single word. Discrete diffusion LLMs invert this paradigm by adapting the iterative denoising math of image generators like Stable Diffusion to discrete text tokens. Starting with a canvas of masked noise tokens, a diffusion LLM predicts and refines dozens or hundreds of tokens simultaneously in a few parallel forward passes.

By turning sequential decoding into parallel matrix operations that saturate GPU Tensor Cores, Inception Labs' Mercury 3 achieves over 1,000 tokens per second on Nvidia H100 hardware across a one-million-token context window—running five to ten times faster than speed-optimized autoregressive counterparts. Beyond raw throughput for real-time voice agents and IDE code completion, bidirectional diffusion refinement allows the model to globally plan and enforce strict JSON schemas or syntactic constraints across an entire output block without left-to-right hallucinations.

Will parallel diffusion LLMs overtake sequential autoregressive decoding for latency-critical AI agents, or will speculative decoding keep traditional Transformers ahead?

Where will discrete diffusion LLMs (dLLMs) have the biggest impact?

0 votos

Comentarios

Sé respetuoso. Los comentarios que reporten varias personas se ocultan hasta que los revise un moderador.

Para leer no necesitas cuenta; solo para comentar. Entras con un código que te llega al correo o con tu cuenta de Google. Mostramos solo el nombre que elijas; nunca tu correo.