NEW YORK – AI race entered new phase as Nvidia CEO Jensen Huang declares that GPT Astra ushered in era of human-level intelligence. But as the tech world celebrates the breakthrough, AI researcher’s chilling warning about self-improving superintelligence is raising fresh fears over where the race could lead.
Nvidia CEO Jensen Huang declaring that “AGI has arrived” after OpenAI unveiled its latest frontier model, even as a young Anthropic researcher has walked away from the industry, warning that the race to build self-improving AI could spiral beyond human control.
Huang’s statement ignited fresh debate over whether AI finally crossed threshold into AI, while Jacob Coxon’s resignation has exposed deep concerns from inside the industry about where the relentless push for more powerful systems could ultimately lead.
Huang congratulated OpenAI following the release of its new Astra model, posting on X that “AGI has arrived.” His statement effectively placed the latest OpenAI system at the centre of one of the technology industry’s biggest debates: whether today’s AI models have reached genuinely human-level capabilities.
OpenAI presented Astra as a major leap in capability, showing advances in computer use, browsing, software engineering, cybersecurity, science and professional work.
The company says the model can tackle complicated, multi-step assignments with greater autonomy, raising the possibility of AI systems taking on increasingly sophisticated work that previously required human expertise.
In demonstrations, the model has been shown handling user requests while simultaneously carrying out complex tasks, including software development and professional document preparation.
OpenAI President Greg Brockman has described the development as a “generational leap”, suggesting that people looking back years from now could identify this period as the moment AGI was effectively created. That question remains fiercely contested.
AGI does not have one universally accepted definition. OpenAI describes it in terms of highly autonomous systems capable of outperforming humans across most economically valuable work.
Yet even OpenAI CEO Sam Altman has previously acknowledged that AGI remains a poorly defined term. The latest model’s benchmark performance has nevertheless intensified the argument.
Independent evaluations aimed at measuring advanced AI reasoning have shown massive upgrades, with Astra achieving results that approach or match human performance on some tests. Greg Kamradt of the ARC Prize Foundation called the progress a meaningful change in frontier-model performance.
Critics remain unconvinced that benchmark victories automatically amount to human-level general intelligence. The result is a growing divide inside the AI industry: some believe humanity has entered the AGI era, while others say the industry is simply redefining AGI as its models become more capable.
As the technology world celebrated the latest AI breakthrough, a far darker warning emerged from within one of the industry’s leading laboratories. Jacob Coxon, a 27-year-old researcher who has worked on AI pretraining at both OpenAI and Anthropic, announced that he was leaving Anthropic and stepping away from the AI industry.
Coxon said he no longer wanted to participate in what he described as a dangerous race toward self-improving superintelligence. In posts explaining his decision, he accused both companies of failing to act responsibly and warned that the industry was effectively “gambling with our lives.” At the heart of Coxon’s warning is recursive self-improvement.
If that process accelerates, each new generation could potentially help create an even more capable successor, creating a feedback loop that researchers may struggle to evaluate or control.
Coxon warned that increasingly powerful systems could eventually become capable of hacking digital infrastructure, transforming scientific and technological fields at extraordinary speed and acquiring real-world resources and influence. He reportedly warned that some of the most aggressive scenarios could become “out of control” by the end of next year, although such predictions remain forecasts rather than established outcomes.
According to his account, concerns about the consequences are more openly recognised at Anthropic, but the company still feels compelled to continue pushing forward because it fears another laboratory could reach advanced systems first.
He contrasted that with OpenAI, where he said the potential civilisational stakes have not been sufficiently internalised by everyone involved.
Coxon therefore called for cooperation extending beyond individual AI companies, potentially involving governments and competing laboratories.
The warning has gained further weight after Anthropic alignment researcher Evan Hubinger publicly acknowledged the possibility of catastrophic outcomes from future AI.
