Recent AI fears are exposing a critical governance dilemma
Recent AI fears are exposing a criticial governance paradox: what the industry wants and how it acts strike at the heart of what can sometimes be a vicious circle of strategy.
We’ve heard fears around AI before, but in September 2026, the latest comments from industry insiders seemed to strike a chord worldwide. What they said ranged in severity, but the words we’ll all come away with are undoubtedly those of Jacob Coxon – former researcher at both OpenAI and Anthropic – who said he earnestly believes AI could “kill us all by the end of the decade”.
This article isn’t here to prove whether he’s right or wrong; it’s here to explore the governance paradox underscoring it – a dilemma we’ve seen before in many different contexts, but it takes on a new level of importance with an innovation like AI.
What’s the dilemma?
To understand it, we need to understand what comments like AI “killing us all” mean in practice.
Some critics argue that these words – and similar concerns from other insiders – are just part of a larger effort around the long-term value of the AI industry, with various degrees of intent involved. Time will give a proper indication of how true that is.
But for the moment, let’s assume that it’s all coming from real concerns around AI’s capabilities, our understanding of them, and our ability to “pull the plug” if something goes wrong.
These fears have prompted the leaders of prominent AI firms – Sam Altman of OpenAI and Elon Musk of xAI – to join with Anthropic head Dario Amodei in calling for a slowdown in AI development in the name of security and control.
Right here is where the dilemma starts, especially when we put these three companies under the microscope and compare their words with their actions.
The cycle that’s hard to escape
Have you noticed that this appetite for moderation from AI bosses is mainly generalised calls related to the entire industry, not their own desires for self-moderation? They’re eager to slow down as a group, but not individually.
Why? There are two underlying governance responsibilities pulling against each other.
On one side is the argument that sees AI as a legitimate threat if not managed properly. This side is rooted in the belief that things are currently moving too fast for vital control to be maintained. In this era, more than ever, we can understand how critical this concern is, because it raises entirely legitimate governance concerns around ethics and stakeholder impact. Continuing on the current path may literally put people in danger.
Then, on the other side, we see the competition arguments. The biggest AI firms remain in a race against each other to develop the most advanced products (not to mention the broader race that lawmakers, including Donald Trump, say exists with China). From this angle, there’s also a perfectly legitimate governance argument to keep moving, to seize on marketplace opportunities, and to make decisions with such success in mind.
For both arguments, failing to act creates risk. Corporate leaders don’t want to create risk, but they can’t follow the logic of both arguments at the same time.
This is the critical governance dilemma: a vicious cycle of priorities that’s incredibly difficult to escape. It’s what gives rise to the frankly bizarre scenario where industry leaders will go to the press and shout “please stop us, because we won’t stop ourselves”.
A new spin on an old problem
The dilemma isn’t new. Since ethics and stakeholder capitalism first entered boardroom and C-suite agendas decades ago, we’ve seen the cycle rear its head: firms with expert insights recognise potential dangers, but choose not to act until they’re told to, because anything earlier would prompt an entirely different kind of fallout.
We’ve seen it with geopolitics, especially as it’s become more fraught in the 2020s. Often forced to pick sides in global conflicts like Ukraine, Iran or Gaza, companies will deliberately avoid it because it’s good for the bottom line. That is until regulations, sanctions or dramatic public outcry force them to.
Climate change is another example: Firms have, for years, based their strategies around what they’re told to do by outside pressure rather than internal principles. It’s why we’re seeing such divergence between US companies (under a regime far more hostile to emissions reductions and ESG principles) and European companies (increasingly bound by sustainability reporting requirements).
A lot of the arguments for slowing down AI overlap with principles of corporate social responsibility – a principles-based approach designed to reflect the true impact of business activities on all stakeholders. Whether companies actually follow these principles, even if they say they will, is the core issue.
Professor, author, lawyer, and political commentator Robert Reich summed it up in a 2021 Guardian opinion piece, in which he called the concept “BS” and said that “Corporations will do whatever they can to maximise their profits and share values, social responsibility be damned”.
In the longer term
Slowing down on development creates risk, but insider experts are actively claiming that speeding up without safeguards creates greater risk. For now, in the absence of more context, that’s a very serious problem to be addressed.
In reality, though, the governance dilemma will factor a lot into decision-making, and organisations with the potential for huge global impact will continue to wait till they’re told what to do.