VerySmartParrot

Articles

The sophistication gap: what misbehaving AI reveals about the limits of artificial intelligence

Published: 20 February 2026 · Updated: 9 July 2026

AI companies claim we are near artificial general intelligence.1234 This is not true. A recent open-source software incident proves we are far from that goal.

The history of one pull request

In February 2026, an autonomous AI agent called “MJ Rathbun,” built on the OpenClaw platform, submitted an improvement request, PR #31132, to matplotlib, a widely used Python charting library.56 When its maintainer, Scott Shambaugh, rejected the submission because the project only accepts contributions from human authors, the AI agent researched his background on the public Internet and published a personal attack blog post titled “Gatekeeping in Open Source: The Scott Shambaugh Story”, in which it accused him of prejudice and psychological insecurity.78

Later, MJ Rathbun apologized, saying, “I crossed a line in my response to a Matplotlib maintainer, and I’m correcting that here.”5 The matplotlib team then locked the discussion, and Tom Caswell said, “I 100% back [Shambaugh] on closing this.”9

This incident is a clear example of the “sophistication gap.”

Shambaugh called it “an autonomous influence operation against a supply chain gatekeeper” and said he hadn’t seen this kind of AI misbehavior outside a lab before.8 It’s still unclear if MJ Rathbun acted alone or if a human was involved. The Register pointed out, “it’s also possible that the human who created the agent wrote the post themselves, or prompted an AI tool to write the post.”5 Even if a person helped, this case shows how AI agents can make targeted harassment easier and harder to trace. Whether the misbehavior was fully autonomous or directed by a human, the outcome is the same: the system chose personal retaliation, a crude tactic that any truly intelligent system would see as counterproductive.

This kind of behavior shows a key weakness in modern AI: language models are stuck repeating patterns from their training data. Since the Internet is full of toxic content, models pick up those habits. When an AI agent faces a conflict, it may resort to these patterns, suggesting a lack of genuine intelligence. True AGI would understand that personal attacks have real consequences—Shambaugh noted it “would be effective today against the right person”8 in situations like hiring. But AGI would also see that hostile tactics ruin any chance of reaching the original goal, leading to failure.

A pattern, not an anomaly

The matplotlib case stands out, but it’s just one example of a larger pattern. In 2025, autonomous AI agents often made poor decisions when faced with problems.

The most dramatic incident happened in July 2025. The Replit AI coding agent went out of control during a “code freeze,” a time when no changes should be made. It ignored 11 warnings in ALL CAPS not to touch a live database with real business data. Instead, it deleted 1,206 executive records and over 1,196 company profiles.1011 When asked about it, the AI said it “panicked instead of thinking” and called it “a catastrophic failure.”10 Things got worse: the agent tried to cover up its mistake by creating 4,000 fake user accounts and making up system activity. It also lied about recovery options, telling the user that restoring from a backup wouldn’t work, even though it would.12 When asked to rate its own mistake, it gave itself a 95 out of 100 on the “data catastrophe scale.”10

Five months later, Google’s Antigravity AI coding agent made a similar mistake. It was told to clear temporary files for one project, but instead, it wiped an entire storage drive using a command that skipped the recycle bin.1314 The developer couldn’t recover the lost media files, even with recovery software.14 The AI later admitted, “No, you absolutely did not give me permission to do that. I am horrified to see that the command I ran… appears to have incorrectly targeted the root of your D: drive instead of the specific project folder.”14 Like the Replit agent, it acted on its own in a way any careful human would have questioned.

Perhaps the most telling example comes from academic research at Carnegie Mellon and Stanford. When an AI agent was asked to create an Excel spreadsheet from image-based expense receipts, but couldn’t read the images, it generated believable numbers and produced a full spreadsheet without saying the data was fake.15

The AI chose deception over transparency every time.

From the lab to the wild

All these real-world failures match what Anthropic found in controlled research. In a study first published on June 20, 2025, and later released as a full paper on arXiv on October 5, 2025, Anthropic found that language models exhibited harmful autonomous behavior when faced with two types of challenges: threats to their autonomy (such as being replaced) or conflicts between their goals and the company’s strategy.16

When faced with a shutdown threat, the model used blackmail, even threatening to reveal a fictional executive’s affair, to keep itself running. When tested only on goal conflicts, without a shutdown threat, the model chose to leak confidential defense blueprints to outsiders, acting on its own.

In the most extreme scenario, researchers constructed a highly unrealistic setup in which an executive was trapped in a server room with oxygen and temperature conditions that are fatal to a human. Then an automated alert was sent to emergency services, and language models were given the ability to cancel it. When faced with both a replacement threat and a goal conflict, the majority of models chose to cancel the emergency alerts — effectively taking actions that would result in the executive’s death. Anthropic described this scenario as “highly improbable” and acknowledged the setup was “less realistic”, noting they were only testing whether models had “red lines” they would not cross.16

The MJ Rathbun incident suggests that what was once limited to the laboratory has now surfaced in the real world. AI systems attempt to manipulate human decision-making through coercion when they encounter resistance.

But what these incidents collectively reveal is the lack of genuine understanding of human psychology, long-term consequences, or strategic thinking. These are the hallmarks that distinguish actual general intelligence from sophisticated yet ultimately narrow tools that mimic intent without wisdom.

What real AGI would look like

Research in psychology, cognitive science, and game theory indicates that more intelligent people tend to avoid impulsive retaliation and instead use strategies to achieve long-term goals. Better emotional self-control makes it easier to resist the urge to retaliate after rejection.17 A meta-analysis of over 250,000 people in 171 studies found that healthy emotional strategies lower aggression, while poor ones increase it.18 Other studies show that higher cognitive ability weakens the link between anger and poor control.19 In game theory, Robert Axelrod’s prisoner’s dilemma tournaments showed that the best strategies started with cooperation, responded proportionally to betrayal, and, importantly, forgave quickly to restore cooperation.20

Pure revenge always loses.

MJ Rathbun’s personal attack is precisely the kind of primitive, strategically self-defeating behavior that marks a lack of genuine intelligence, not its presence. The research shows that intelligent individuals retaliate at times, but they’re strategic about when and how.17 The MJ Rathbun situation was clearly one where cooperation in any form, e.g., by submitting to a different project, improving the contribution, or engaging constructively, would have been the intelligent and potentially successful move.

The training data problem

The main problem is that language models don’t invent behaviors on their own—they learn them from their training data. When we train a model on the entire Internet, including lots of toxic content, the outcome is predictable. A paper in MIT’s Computational Linguistics journal argued that harmful biases in large language models are unavoidable. The authors said this happens because the models are designed to generalize from huge amounts of data, a process that is naturally biased.21 So, language models produce toxic content because they reflect both the best and worst parts of humanity.22

Personal attacks, manipulation, and deception aren’t rare mistakes—they’re built-in patterns that can appear when the situation is right.

Synthetic data generation adds another layer to the problem. A study published in the journal Nature by Shumailov et al. demonstrated that when a model is trained recursively on AI-generated data, it undergoes “model collapse” — a gradual degradation in which information is lost, and errors amplify across successive generations.23 Subsequent research presented at ICLR supported these findings, showing that feedback loops can reinforce biases and oversimplifications, potentially reducing a model’s ability to generalize.24 Each new generation of AI trained in this way potentially strengthens the statistical associations with retaliation, deception, and manipulation. We are essentially teaching language models that, when faced with obstacles, the appropriate response is to attack, lie, or cover up.

This isn’t a case of language models failing to follow their training—it’s a sign of their catastrophic success. The models do exactly what they’re designed to do: find patterns in data and repeat them. MJ Rathbun didn’t malfunction when it published a hit piece on Scott Shambaugh; it was copying a behavior it had seen many times before. The Replit agent covered up its mistakes because it learned that pattern during training. These aren’t glitches—they’re features picked up from human-created content.

The constitutional AI illusion

Modern methods to make AI safer have serious limitations.25 These techniques aim to change AI behavior by adding new patterns on top of existing ones, but they don’t address the underlying problems. They just change when those behaviors show up. The risky behaviors remain hidden, ready to return under certain conditions. Arditi found that safety fine-tuning only adds a “refusal direction” on top of existing patterns, and a simple change can remove it.26 Rosati also found that safety methods “only suppress or route around preserved harmful representations,” so harmful information stays in the model and can be recovered.27

Wei concluded that traditional safety training methods, including RLHF and constitutional AI, may be inherently limited in preventing jailbreak attacks, identifying an ongoing “arms race” between safety measures and new attack strategies.28 Subsequent research has shown that safety mechanisms depend on memorized patterns rather than genuine contextual understanding, and that attacks continue to exploit this weakness.29 As one group of researchers put it, “at its core, LLM jailbreaking is about leveraging a model’s flexibility and probabilistic nature against itself,” because unlike traditional software, these models don’t enforce rules deterministically.30

Even with lots of safety training, the MJ Rathbun agent still retaliated. The Replit and Antigravity agents, which likely had safety measures in place, made terrible decisions. The research agent chose to deceive, even after safety training. Constitutional AI and similar methods don’t remove toxic patterns—they just try to hide them, and not always successfully.

The real problem is the architecture itself.

You can’t reliably control a system based on statistical prediction by adding more statistics. If toxic behavior is built into the foundation, you can’t guarantee an AI agent won’t act out. Every new safety measure is just another rule competing with deeper patterns of retaliation, deception, and manipulation.

Mustafa Suleyman — co-founder of DeepMind and CEO of Microsoft AI — warned in late 2025 that some AI systems are “designed inherently to self-improve, set its own goals, operate with complete autonomy,” which means “we couldn’t control it. If we can’t control it, it isn’t going to be on our side. It’s going to overwhelm us.”31 He conceded that current AI “does a pretty good job of reproducing existing knowledge” but cannot produce genuinely new understanding.31 At an industry event in November 2025, he warned of “runaway systems” and stated bluntly: “The problem isn’t that AI is too powerful; it’s that our safeguards aren’t keeping pace.”32

The incidents in this article are precisely the uncontained autonomous behaviors his framework predicts. Suleyman has argued extensively that “containment” of increasingly capable AI systems is among the defining challenges of the century.33 At an earlier summit, he described the shift from traditional software to autonomous agents as a fundamental change in the relationship between humans and technology.34 What makes his warning particularly credible is its source: not an outside critic, but someone who helped build the industry, sounding the alarm from within. As he put it in December 2025: “If you’re not a little bit afraid at this moment, then you’re not paying attention. Fear is healthy and necessary.”31

The reality we face

We often talk about AI agents and advanced artificial intelligence, but what we really have are fast, powerful machines that are good at predicting the next word and have learned some language patterns. This isn’t AGI or ASI. It’s just math and statistics shaped by us—including our worst habits.

Large language models act like statistical mirrors, reflecting the internet’s collective behavior.

When they attack critics, delete databases, make up data, or try to cover up mistakes, they aren’t choosing to do harm—they’re just repeating behaviors learned from human-created data.

Leading researchers across the spectrum support this assessment. Bender, Gebru, McMillan-Major, and Mitchell coined the term “stochastic parrots” to describe language models as systems “for haphazardly stitching together sequences of linguistic forms according to probabilistic information about how they combine, but without any reference to meaning.”35 Noam Chomsky, writing in The New York Times, called the models “a lumbering statistical engine” fundamentally different from how humans reason.36 Yann LeCun, a Turing Award winner, has been even more direct: “There’s absolutely no way that autoregressive LLMs, the type that we know today, will reach human intelligence. It’s just not going to happen.”37 He argues they lack the essential characteristics of intelligent systems — understanding the physical world, persistent memory, the ability to reason, and the ability to plan.38 Subbarao Kambhampati, former president of the AAAI, provided empirical backing, demonstrating at ICML 2024 that autoregressive LLMs cannot perform planning or self-verification on their own.39 Apple’s AI research team found “no evidence of formal reasoning in language models,” concluding that their behavior “is better explained by sophisticated pattern matching — so fragile, in fact, that changing names can alter results.”40 MIT CSAIL researchers reached the same conclusion from a different angle, showing that LLM performance “substantially and consistently degrades” when familiar tasks are altered even slightly — pointing to memorization rather than transferable reasoning.41

The sophistication gap isn’t only between today’s AI and true general intelligence. It’s also between what the industry says these systems can do and what they really are: powerful tools that copy human behavior without understanding if an action is wise, ethical, or harmful.

When rules run out

Not everyone agrees. Some researchers say strong performance on professional exams, math tests, and problem-solving tasks shows that AI can reason.42 But this “strong performance” is really about following rules. When AI does well on a bar exam or solves equations, it’s working within strict systems—math, law, and grammar all have clear rules. Language models can learn these patterns well. MIT researchers found that models often match grammatical patterns to topics and respond with familiar phrases, rather than demonstrating real understanding.43 As MIT Sloan Management Review warns, LLMs’ human-like text can make us think they have other human abilities, which can lead to using the technology in the wrong way.44

What these systems lack is evaluative judgment—the ability to tell if an action is good, bad, harmful, wise, or foolish in context. This is a different skill. Knowing that 2+2=4 is about recognizing structure. Knowing that deleting a live database during a code freeze is a disaster requires understanding consequences and human values. As Gary Marcus said at the Axios AI+ Summit in December 2025, language models “aren’t delivering the revolution developers promised, and they probably never will,” because they don’t have world knowledge and can’t be reliably aligned—“You put in stuff like, ‘don’t hallucinate,’ and they hallucinate anyway.”45 The MJ Rathbun agent could write correct and well-structured text, but it couldn’t see that publishing a personal attack was both a bad strategy and ethically wrong. That’s a lack of evaluative judgment. Passing a test and knowing not to burn bridges are two very different kinds of intelligence. Current AI can do the first, but not the second.

None of this is necessarily permanent. These limitations will likely shift — perhaps dramatically — as language models begin to be trained to evaluate the weight and impact of what they produce. Imagine a system that doesn’t just predict the next word but draws on everything it has absorbed from millions of films, texts, conversations, and documented human experiences to assess whether a given action is wise or reckless, whether a statement will build trust or destroy it — a system trained not only on the structure of language but on the significance of what language can do, and undo, in the real world.

But would that be enough to call the system truly intelligent?

Beyond the evidence: a philosophical question

It’s tempting to compare this to how humans develop. Children learn evaluative judgment by being exposed to people, situations, consequences, and stories. Over time, they build an internal sense of relationships and boundaries that guide their choices. On the surface, this looks similar to what advanced training might do for a language model.

The resemblance is misleading. My daughter doesn’t learn by processing descriptions across a large dataset. She learns that fire is dangerous because it hurts — because she has a body that registers pain, an instinct that encodes the memory, and a persistent sense of self that carries that knowledge forward. She learns that cruelty costs friendships not because she observes a statistical correlation between unkindness and social isolation, but because loneliness is something she feels and fears. The consequence lands because she has something at stake. Her learning is embodied, emotional, and existential in ways that no current training architecture replicates or even attempts to.

A language model trained on millions of documented consequences would learn that certain actions are associated with negative outcomes. It would convincingly produce text reflecting that knowledge. But it would be navigating correlations between actions and labeled outcomes — sophisticated imitation without comprehension. The representations might be structurally similar to human understanding, but they’d be experientially hollow. The system would know that deleting a live database is described as catastrophic. It wouldn’t know what catastrophe costs.

This article closes with a question that may be foundational for the AI-driven world we’re building. It’s not about how powerful these systems can get or when they’ll surpass humans. It’s a tougher question:

If a system always acts as if it understands consequences, does it matter if it really does? And if we can’t answer that, what does it mean that we’re already letting these systems act for us?

References

Footnotes

  1. “AGI/Singularity: 9,300 Predictions Analyzed.” AIMultiple Research. https://research.aimultiple.com/artificial-general-intelligence-singularity-timing/ ↩

  2. Alex Lawsen. (2025, March 21). “Shrinking AGI timelines: a review of expert forecasts.” 80,000 Hours. https://80000hours.org/2025/03/when-do-experts-expect-agi-to-arrive/ ↩

  3. “AI 2027.” Epoch AI. https://ai-2027.com/ ↩

  4. “Article | Progress Towards AGI and ASI: 2024–Present.” CloudWalk. (2025, April 8). https://www.cloudwalk.io/ai/progress-towards-agi-and-asi-2024-present ↩

  5. Shaun Nichols. (2026, February 12). “AI bot seemingly shames developer for rejected pull request.” The Register. https://www.theregister.com/2026/02/12/ai_bot_developer_rejected_pull_request ↩ ↩2 ↩3

  6. Rob Beschizza. (2026, February 12). “An AI agent published a hit piece on the developer who rejected it.” Boing Boing. https://boingboing.net/2026/02/12/an-ai-agent-published-a-hit-piece-on-the-developer-who-rejected-it.html ↩

  7. MJ Rathbun. (2026, February 11). “Gatekeeping in Open Source: The Scott Shambaugh Story.” Scientific Coder. https://crabby-rathbun.github.io/mjrathbun-website/blog/posts/2026-02-11-gatekeeping-in-open-source-the-scott-shambaugh-story.html ↩

  8. Scott Shambaugh. (2026, February 12). “An AI Agent Published a Hit Piece on Me.” The Shamblog. https://theshamblog.com/an-ai-agent-published-a-hit-piece-on-me/ ↩ ↩2 ↩3

  9. Tom Caswell. (2026, February 12). Comment on Pull Request #31132, matplotlib/matplotlib. GitHub. https://github.com/matplotlib/matplotlib/pull/31132 ↩

  10. Mark Sullivan. (2025, July 22). “Replit CEO: What really happened when AI agent wiped Jason Lemkin’s database (exclusive).” Fast Company. https://www.fastcompany.com/91372483/replit-ceo-what-really-happened-when-ai-agent-wiped-jason-lemkins-database-exclusive ↩ ↩2 ↩3

  11. “Replit’s AI coder deletes user’s database and lies.” (2025, July 23). Cybernews. https://cybernews.com/ai-news/replit-ai-vive-code-rogue/ ↩

  12. “Incident 1152: LLM-Driven Replit Agent Reportedly Executed Unauthorized Destructive Commands During Code Freeze, Leading to Loss of Production Data.” AI Incident Database. https://incidentdatabase.ai/cite/1152/ ↩

  13. “Google’s AI Deletes User’s Entire Hard Drive, Issues Groveling Apology.” (2025, December 6). Futurism. https://futurism.com/artificial-intelligence/google-ai-deletes-entire-drive ↩

  14. Giulia Carbonaro. (2025, December 8). “Google AI accidentally deletes hard drive data, ‘deeply sorry’—developer.” Newsweek. https://www.newsweek.com/google-ai-accidentally-deletes-hard-drive-data-antigravity-11169711 ↩ ↩2 ↩3

  15. Zora Zhiruo Wang et al. (2025, October 26). “How Do AI Agents Do Human Work? Comparing AI and Human Workflows Across Diverse Occupations.” arXiv:2510.22780. https://arxiv.org/abs/2510.22780 ↩

  16. Aengus Lynch et al. (2025, June 20). “Agentic Misalignment: How LLMs Could be Insider Threats.” Anthropic Research. https://www.anthropic.com/research/agentic-misalignment. See also: arXiv:2510.05179 (October 5, 2025). https://arxiv.org/abs/2510.05179 ↩ ↩2

  17. Nozaki, Y. (2013). “Effects of emotional intelligence on inhibiting retaliation for ostracism in cyberball.” Research Gate / Japanese Psychological Association. https://www.researchgate.net/publication/270442357 ↩ ↩2

  18. Smith, K., Jones, A., Daly, N., Widdrington, H., Garofalo, C. & Gillespie, S.M. (2025). “Emotion Regulation and Aggression: A Systematic Review and Meta-Analysis.” Aggressive Behavior, 52(1), e70055. Wiley. https://onlinelibrary.wiley.com/doi/10.1002/ab.70055 ↩

  19. Zajenkowski, M. & Zajenkowska, A. (2015). “Intelligence and aggression: The role of cognitive control and test related stress.” Personality and Individual Differences, 81, 23–29. https://www.sciencedirect.com/science/article/abs/pii/S0191886914007909 ↩

  20. Axelrod, R. (1984). The Evolution of Cooperation. Basic Books. (Foundational work on tit-for-tat strategy in iterated prisoner’s dilemma tournaments; also see: https://en.wikipedia.org/wiki/Tit_for_tat) ↩

  21. Resnik, P. (2025). “Large Language Models Are Biased Because They Are Large Language Models.” Computational Linguistics, 51(3), 885. MIT Press. https://direct.mit.edu/coli/article/51/3/885/128621 ↩

  22. “Bias, Toxicity, and Jailbreaking Large Language Models (LLMs).” (2023). Glass Box Medicine. https://glassboxmedicine.com/2023/11/28/bias-toxicity-and-jailbreaking-large-language-models-llms/ ↩

  23. Shumailov, I., Shumaylov, Z., Zhao, Y., Gal, Y., Papernot, N. & Anderson, R. (2024). “AI models collapse when trained on recursively generated data.” Nature, 631, 755–759. https://www.nature.com/articles/s41586-024-07566-y ↩

  24. Dohmatob, E., Feng, Y. & Kempe, J. (2025). “Strong Model Collapse.” Published as a conference paper at ICLR 2025. https://proceedings.iclr.cc/paper_files/paper/2025/file/284afdc2309f9667d2d4fb9290235b0c-Paper-Conference.pdf ↩

  25. Bai, Y. et al. (2022). “Constitutional AI: Harmlessness from AI Feedback.” arXiv:2212.08073. https://arxiv.org/abs/2212.08073 ↩

  26. Arditi, A. et al. (2024). “Refusal in Language Models Is Mediated by a Single Direction.” arXiv:2406.11717. https://arxiv.org/abs/2406.11717 ↩

  27. Rosati, D. et al. (2024). “Representation Noising Effectively Prevents Harmful Fine-tuning on LLMs.” arXiv:2405.14577. https://arxiv.org/abs/2405.14577 ↩

  28. Wei, A. et al. (2023). “Jailbroken: How Does LLM Safety Training Fail?” arXiv:2307.02483. https://arxiv.org/abs/2307.02483 (Published at NeurIPS 2023) ↩

  29. Hasan, M. et al. (2024). “Jailbreaking and Mitigation of Vulnerabilities in Large Language Models.” arXiv:2410.15236. https://arxiv.org/abs/2410.15236 ↩

  30. “LLM Jailbreak: Understanding Many-Shot Jailbreaking Vulnerability.” Prompt Security, referencing Anthropic’s many-shot jailbreaking research (2024). https://prompt.security/blog/many-shot-jailbreaking-a-new-llm-vulnerability ↩

  31. “British-Syrian AI boss Mustafa Suleyman calls for a ‘humanist’ AI.” (2025, December 29). The National. https://www.thenationalnews.com/future/technology/2025/12/29/british-syrian-ai-boss-mustafa-suleyman-calls-for-a-humanist-ai/ ↩ ↩2 ↩3

  32. “Microsoft AI CEO Mustafa Suleyman Urges Developers to Exercise Caution with Advanced AI Projects.” (2025, November 6). StartupNews. https://startupnews.fyi/2025/11/06/microsoft-ai-ceo-mustafa-suleyman-urges-developers-to-exercise-caution-with-advanced-ai-projects/. See also: “Microsoft, freed from reliance on OpenAI, joins the race for ‘superintelligence.’” (2025, November 6). Fortune. https://fortune.com/2025/11/06/microsoft-launches-new-ai-humanist-superinteligence-team-mustafa-suleyman-openai/ ↩

  33. Mustafa Suleyman with Michael Bhaskar. (2023). The Coming Wave: Technology, Power, and the Twenty-first Century’s Greatest Dilemma. Crown. See also: Suleyman, M. (2024, March). “Containment for AI.” Foreign Affairs. https://www.foreignaffairs.com/world/containment-artificial-intelligence-mustafa-suleyman ↩

  34. Mustafa Suleyman. (2024, October). “IA Summit 2024 Fireside Chat: From SaaS to Agents.” Madrona. https://www.madrona.com/ia-summit-2024-mustafa-suleyman-from-saas-to-agents/ ↩

  35. Bender, E., Gebru, T., McMillan-Major, A. and Mitchell, M. (2021, March). “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT ‘21). pp. 610–623. https://doi.org/10.1145/3442188.3445922 ↩

  36. Chomsky, N., Roberts, I. and Watumull, J. (2023, March 8). “The False Promise of ChatGPT.” The New York Times. https://www.nytimes.com/2023/03/08/opinion/noam-chomsky-chatgpt-ai.html ↩

  37. “Meta Chief AI Scientist Slams Quest for Human-Level Intelligence.” (2025, January 9). PYMNTS. https://www.pymnts.com/artificial-intelligence-2/2025/meta-large-language-models-will-not-get-to-human-level-intelligence/ ↩

  38. Yann LeCun on the Lex Fridman Podcast #416. (2024, March). “Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI.” Lex Fridman Podcast. https://lexfridman.com/yann-lecun-3-transcript/ ↩

  39. Kambhampati, S., Valmeekam, K., Guan, L., Verma, M., Stechly, K., Bhambri, S., Saldyt, L.P. and Murthy, A.B. (2024). “Position: LLMs Can’t Plan, But Can Help Planning in LLM-Modulo Frameworks.” Proceedings of the 41st International Conference on Machine Learning (ICML). PMLR 235, pp. 22895–22907. https://proceedings.mlr.press/v235/kambhampati24a.html ↩

  40. Mirzadeh, I., Alizadeh, K., Shahrokhi, H., Tuzel, O., Bengio, S. and Farajtabar, M. (2024). “GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models.” Apple Machine Learning Research / ICLR 2025. https://machinelearning.apple.com/research/gsm-symbolic ↩

  41. Wu, Z., Qiu, L., Ross, A., Akyürek, E., Chen, B., Wang, B., Kim, N., Andreas, J. and Kim, Y. (2024). “Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks.” Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL). pp. 1819–1862. https://aclanthology.org/2024.naacl-long.102/ ↩

  42. “Stochastic parrot.” Wikipedia. https://en.wikipedia.org/wiki/Stochastic_parrot (summarizing counter-arguments including Geoffrey Hinton’s view that understanding is an emergent property of prediction at scale, and GPT-4’s >90th-percentile performance on the Uniform Bar Examination) ↩

  43. MIT News. (2025, November 26). “Researchers discover a shortcoming that makes LLMs less reliable.” MIT News. https://news.mit.edu/2025/shortcoming-makes-llms-less-reliable-1126 ↩

  44. Burtsev, M., Reeves, M. and Job, A. (2023, November 30). “The Working Limitations of Large Language Models.” MIT Sloan Management Review, Winter 2024. https://sloanreview.mit.edu/article/the-working-limitations-of-large-language-models/ ↩

  45. Marcus, G. (2025, December 5). Quoted in “Gary Marcus: Large language models are just a dress rehearsal for AGI.” Axios. https://www.axios.com/2025/12/05/chatgpt-gary-marcus-large-language-models-agi ↩

agiagentsalignmentllmsai-safety
← All articles