Game Over for Pure LLMs: Even Turing Award Winner Rich Sutton Has Left the Bus Overview Gary Marcus discusses a significant shift in AI thought leadership: Rich Sutton, a Turing Award winner and advocate for scaling AI through large language models (LLMs), has publicly critiqued the sufficiency of pure LLM approaches. This marks a turning point where one by one, leading AI figures have turned skeptical about the current predominant focus on scaling LLMs. Rich Sutton and The Bitter Lesson Sutton authored The Bitter Lesson (2019), an influential essay arguing that AI progress depends on general-purpose methods that scale well, rather than hand-engineered solutions. The essay is a favorite among scaling advocates, especially its emphasis on the power of methods that continue to improve as computational resources increase. Despite its influence, Gary Marcus considered this view overstated and flawed, planning to critique it publicly. Sutton’s Shift in View Recently, Sutton expressed reservations about the sufficiency of pure LLM scaling on a popular podcast. His statements closely mirror Gary Marcus’ long-standing critique: that pure prediction-based systems (LLMs) lack true understanding and world models. Both Sutton and Marcus agree on the need for incorporating world models and recognize the limitations of pure prediction. Differences exist in their preferred methods: Sutton favors reinforcement learning while Marcus advocates for neurosymbolic approaches and innate constraints. Broader Context: The Changing Landscape Several prominent AI thinkers have become critical of pure LLM approaches: Yann LeCun voiced a similar critique by the end of 2022. Demis Hassabis, Nobel Laureate & DeepMind CEO, also expresses reservations. The narrative that scaling LLMs alone is enough is largely held by a shrinking group, often dismissed as grifters by Marcus. Implications and Next Steps There is consensus on the problems with pure LLMs but no agreement yet on precise solutions. Marcus encourages investing in alternatives with world models, neurosymbolic AI, and hybrid approaches, suggesting that even small investments could unlock breakthroughs. He views the current enthusiasm for scaling LLMs without architectural innovation as a dead end. Community Reactions Comments reflect agreement with the cautious stance on scaling: Some point out that multimodal data (images, videos) still relies on human-generated content, limiting foundational models' ability to grasp fundamental concepts like physics or human behavior. There is recognition that no major breakthroughs are currently evident and progress feels incremental. Conclusion Gary Marcus frames the turning away of Rich Sutton and other AI leaders from a pure scaling paradigm as a critical moment in AI research. This shift signals the end of the "game" where scaling LLMs was seen as "all you need," urging the community to explore new directions incorporating deeper modeling and hybrid methods. --- For deeper insights and ongoing updates, subscribing to Gary Marcus's newsletter is recommended.