Anthropic Chief Warns Powerful AI Could Create New Public Safety Risks
Anthropic CEO Dario Amodei has called for advanced artificial intelligence systems to be regulated in a manner similar to aircraft and pharmaceutical products, arguing that frontier AI models are rapidly approaching a level of capability that could create significant public safety and national security challenges.
In his essay, Policy on the AI Exponential, Amodei said artificial intelligence is advancing far faster than governments, regulators, and policymakers can adapt. He warned that within the next few years, the world could witness the arrival of what he describes as “Powerful AI” systems with capabilities comparable to “a country of geniuses in a datacenter.”
According to Amodei, the gap between technological progress and regulatory preparedness is widening, making it increasingly important for governments to establish frameworks capable of managing both the opportunities and risks associated with advanced AI.
Why Does Amodei Believe AI Needs Stronger Regulation?
Amodei argues that evidence of AI’s growing power is now impossible to ignore. Frontier models are already demonstrating sophisticated capabilities across cybersecurity, scientific research, software development, and complex problem solving.
While these capabilities can accelerate innovation and productivity, they also introduce new risks. Amodei believes advanced AI systems could potentially be used to target critical infrastructure, disrupt financial systems, or create national security vulnerabilities if adequate safeguards are not implemented.
He further warned that cybersecurity threats may only represent the first wave of challenges. As AI systems become increasingly capable, risks related to biological research, autonomous decision-making, and self-directed AI behavior could emerge.
According to Amodei, transparency alone is no longer sufficient. AI has evolved into a technology category that requires active oversight, similar to industries where failures can directly affect public safety.
A Regulatory Framework Similar To Aviation
To address these concerns, Amodei proposed a regulatory model inspired by agencies such as the US Federal Aviation Administration (FAA), which oversees aircraft certification and operational safety.
Under his proposal, advanced AI models exceeding defined computational thresholds would undergo mandatory testing and risk assessments before public deployment. These evaluations would examine areas including cybersecurity threats, biosecurity concerns, AI autonomy, and automated research capabilities.
Governments would have the authority to delay or restrict the release of models deemed unsafe, while AI developers would be required to implement rigorous security controls, conduct red-team testing, and maintain continuous monitoring systems.
Independent evaluators or accredited regulatory agencies could oversee the process, ensuring that safety assessments remain credible and transparent.
AI Could Transform Economies And Labour Markets
Beyond safety concerns, Amodei believes artificial intelligence could fundamentally reshape global labour markets and economic structures.
He argues that future AI systems may eventually outperform humans across many cognitive tasks, driving unprecedented economic growth while simultaneously increasing inequality. In such a scenario, societies may face a new challenge: ensuring that the benefits of AI-driven productivity are distributed broadly rather than concentrated among a small group of companies and individuals.
To prepare for this possibility, Amodei advocates better tracking of AI-related job displacement, policies that encourage employment retention, long-term income support mechanisms, and broader economic frameworks capable of supporting workers during periods of transition.
He also suggested exploring universal capital accounts as a way to distribute the economic gains generated by advanced AI systems.
Concerns Around Democracy, Surveillance And Civil Liberties
Amodei warned that powerful AI systems could become highly effective tools for authoritarian governments if appropriate safeguards are not established.
He believes AI-enabled surveillance could allow governments to analyze enormous volumes of data and infer deeply personal information about citizens, creating privacy challenges beyond the scope of current legal protections.
The Anthropic CEO also raised concerns about autonomous weapons systems, arguing that future military technologies could enable governments to exercise force with reduced human oversight.
To protect democratic institutions, Amodei proposed clear accountability standards for autonomous weapons, restrictions on domestic deployment of such systems, tighter controls on large-scale data collection, and broader access to AI tools for citizens.
He emphasized that AI should not be controlled solely by governments or corporations and called for balanced systems of oversight and accountability.
AI Is Becoming A Strategic Geopolitical Asset
Amodei believes artificial intelligence will become one of the defining geopolitical technologies of the coming decades.
According to him, countries that successfully develop and deploy advanced AI capabilities could gain strategic advantages comparable to those achieved through major military or technological breakthroughs in previous eras.
To address this reality, he proposed stronger cooperation among democratic nations through shared AI principles, coordinated semiconductor supply chains, collaborative risk management, and mutual support against adversarial AI systems.
Such cooperation, he argues, would help ensure that the benefits of AI are distributed among allied nations while limiting the technology’s potential misuse.
Existing Regulatory Systems May Struggle To Keep Pace
Amodei also highlighted the possibility that AI could overwhelm existing regulatory systems across multiple industries.
In healthcare, for example, AI has the potential to dramatically accelerate drug discovery, medical research, and treatment development. However, approval processes at agencies such as the US Food and Drug Administration (FDA) and the European Medicines Agency (EMA) remain designed for a much slower pace of innovation.
Without modernization, he warned, regulatory bottlenecks could prevent society from fully benefiting from AI-driven scientific breakthroughs.
As a result, Amodei believes regulators should begin developing standards now for evaluating AI-assisted research so that validated innovations can be adopted more quickly in the future.
Why Anthropic Is Pushing For Stronger Oversight
Anthropic has increasingly positioned itself as a leading advocate for AI safety and responsible governance. The company has supported proposals related to frontier model testing, transparency requirements, and research into the economic impact of AI-driven automation.
For Amodei, the challenge is not whether AI will transform society, but whether governments and institutions can adapt quickly enough to maximize its benefits while minimizing its risks.
As AI capabilities continue to advance at an unprecedented pace, his call for aviation-style oversight reflects a growing belief within parts of the industry that artificial intelligence should be treated as critical infrastructure rather than just another software product.
