• TRM$ 3.461,92 +1,37 % +$ 46,82TRM oficial (Superintendencia Financiera), vigente del 21 nov al 23 nov
  • Euro$ 3.804,64 +1,37 % +$ 51,45Referencia: TRM cruzada con la tasa del Banco Central Europeo del 20 nov; no es la cotización de mercado
  • Peso mexicano$ 181,56 +1,37 % +$ 2,45Referencia: TRM cruzada con la tasa del Banco Central Europeo del 20 nov; no es la cotización de mercado
← Volver a la comunidad
Tecnología

Why is recursive training on synthetic tokens triggering model collapse as publishers block AI web scrapers?

Frontier AI labs have exhausted the open internet's supply of pristine human text just as digital publishers erect hard paywalls against web crawlers. Forced to train next-generation models on synthetic, AI-generated tokens, developers now face recursive model collapse that erodes reasoning variance and factual accuracy.

To secure uncorrupted human tokens, hyperscalers have signed nine-figure licensing deals—including OpenAI's $250 million agreement with News Corp, Meta's $50 million annual media contracts, and Google's API partnerships with Reddit. Yet because generative search summaries like AI Overviews have cannibalized organic referral traffic by up to 40%, we are seeing the long tail of independent publishers deploy Cloudflare bot-blockers and robots.txt restrictions to starve uncompensated scrapers.

When engineering teams substitute scarce human prose with recursive synthetic pre-training data, the probability distribution of the model's weights truncates low-probability linguistic tails—a statistical decay known as model collapse. While post-training Reinforcement Learning with Verifiable Rewards (RLVR) helps preserve coding and math benchmarks, models fed on synthetic echo chambers suffer higher hallucination rates on out-of-distribution enterprise reasoning tasks.

Can reinforcement learning and verifiable synthetic data overcome the human token shortage, or will proprietary human archives become the ultimate moat in AI?

What will solve the human training data shortage for frontier AI?

0 votos

Comentarios

Sé respetuoso. Los comentarios que reporten varias personas se ocultan hasta que los revise un moderador.

Para leer no necesitas cuenta; solo para comentar. Entras con un código que te llega al correo o con tu cuenta de Google. Mostramos solo el nombre que elijas; nunca tu correo.