A viral post by Jacob Coxon on September 8 ignited a global debate on AI risks, with experts from top AI firms warning of potential catastrophe. Coxon, a 27-year-old British researcher, resigned from Anthropic after claiming companies like OpenAI and Anthropic were racing toward “self-improving superintelligence” and gambling with human lives; his post was viewed 153 million times within 36 hours, prompting widespread concern and action from both AI labs and politicians.

Experts Sound Alarms on AI Extinction Risk

Evan Hubinger, an Anthropic researcher, emphasized that AI could potentially kill all humans unless there is regulation or a coordinated slowdown. Marcus Williams of OpenAI estimated a 70% risk of human extinction within the next few years without such measures. Anthropic CEO Dario Amodei published a 3,800-word essay on September 12, warning that misaligned AIs could take over the internet in as little as six to 12 months.

Two of Amodei’s rivals, Elon Musk of SpaceXAI and Sam Altman of OpenAI, agreed that slowing down AI development was necessary. Altman announced that OpenAI would not go public in 2024 due to safety concerns. Both leaders said they would allow independent safety experts into their companies but stopped short of halting model training.

Varied Reactions to AI Risks

Gary Marcus, a New York University professor and AI skeptic, acknowledged the seriousness of Amodei’s warnings but urged caution. Marcus argued that claims of AI taking over the internet or causing massive damage are speculative and require scrutiny. He called some statements “nonsensical” and emphasized that AI’s risks are not inevitable if approached responsibly.

Other experts, however, are more alarmist. One warning cited by South Korean media described a scenario where a “swarm” of AI agents could control a persistent botnet,essentially a network of hijacked devices—and take over the internet, potentially causing trillions in damage. Gary Marcus called this scenario possible but insisted it is not a given outcome.

Public and Political Response to AI Concerns

The public response has been swift. Dozens of politicians joined the conversation on AI risks within days of Coxon’s post. The debate has shifted from theoretical concerns to urgent calls for regulation. While AI companies have taken steps to involve external safety experts, they have not yet agreed to pause or slow down development of new models.

As of now, the future of AI remains uncertain. While some warn of existential threats, others argue for a measured and cautious approach. The next few months will be key in determining how the world responds to AI’s growing power and the potential risks it poses to humanity.