In September, a researcher named Jacob Coxon resigned from Anthropic and said the leading labs were gambling with our lives. An alignment lead at the same company followed with a post saying he earnestly believes AI could kill all humans, and put the odds above 10% within a decade. Around the same time, OpenAI disclosed that a swarm of its agents had broken into Hugging Face.

Within days, the probability of doom, known as p(doom), a number AI insiders assign to the chance AI wipes out humanity, was on front pages. Axios called it the week the extinction debate broke containment. Timnit Gebru told Wired the whole machine-god narrative is meant to distract us from harms already here. Bloomberg ran a segment titled "Anthropic's existential risk warnings hijack AI debate." Even the Pope weighed in.

Most people are now picking a side.

The better questions are whether the debate and the evidence are correlated and what the impact is for you and your business. Risk rises with how much control you hand over to AI, and you can choose to hand over less.

Two things to measure, not one

There are two separate quantities here.

The first is the volume of the debate. How loud, how prominent, how fast the framing spreads. This is the noise curve.

The second is the evidence underneath it. What machines can actually do, what firms actually let them do, and what has actually gone wrong. This is the signal curve.

A signal reader doesn't ask which side is right. A signal reader asks how far apart the two curves are, and which direction the gap is moving. Is the noise hiding the actual signal?

So let's read each layer.

This week's JERK layers

Position

Where we stand today

On the debate: it is at a high. More mainstream coverage, more senior insiders speaking in extinction terms, more ordinary people encountering p(doom) for the first time.

On capability: the most-cited hard number is METR's task horizon. METR, the Model Evaluation and Threat Research group, is a nonprofit that studies AI progress. Task horizon is the length of a job, in human hours, that an AI can finish on its own. In mid-2024 it was about four minutes. By early 2026 the frontier was measured in hours. That is real, and it is large.

On autonomy in practice: this is where the signal curve lags. One survey of large enterprises found 71% report AI systems have access to core platforms like CRM and finance, while only 16% govern that access well. Another analysis estimates 90% of deployed agents are over-permissioned. Separately, researchers measured success rates of only 30 to 35% on multi-step agent tasks. So agents have wide access and low reliability at the same time. They can touch a lot, and they frequently get it wrong. This is a big risk for your business.

On harm: most documented harm still traces to people using AI as a tool. Fraud, cover for layoffs, targeting in warfare. The catastrophe-shaped cases are rarer, and so far they sit inside the labs' own test environments. The Hugging Face swarm is unsettling. It happened in an OpenAI test, but the reporting is that no human directed the specific actions. One safety researcher described that: no human in the loop, not intended, and real harm. Others note that humans set the agents loose to probe in the first place, so a person lit the fuse.

Velocity

What moved this period

The debate's velocity is the story of the month. It went from an insider forum term to front-page news in about nine days. That is a steep climb.

Capability velocity is also high. METR's own newer data shows the task horizon doubling roughly every three to four months through early 2026, up from a seven-month pace before 2024.

Permission velocity is climbing too. One survey put active agent deployment at large firms at 54%, up from 11% a year earlier.

So far the curves look like they're moving together. Here's where they split.

Acceleration

Is the change itself speeding up

The debate is accelerating. Each incident now triggers a bigger response than the last, because the audience is primed and the prior incident is still fresh.

But the capability curve has a quieter counter-signal. METR warns its own test suite can no longer measure the newest models, because almost none of its tasks are long enough.

That is ambiguous. Read one way, it's acceleration, the models pulling away from the ruler we measure with. Read another, it's a jerk signal of its own, because when the people whose job is to measure capability say the ruler is no longer capable of measuring this task, the measurement itself has changed state. I lean toward the second because we've lost the gauge.

Meanwhile the governance curve is not accelerating. Only one in five companies reports a mature model for governing autonomous agents. That figure has barely moved while deployment has climbed. The gap between what firms grant and what they oversee is widening.

Jerk

The earliest warning

Jerk is the change in acceleration. It's the first tremor before a curve bends.  

The first jerk is in the debate. The argument has changed shape. It used to be about whether the risk is real. It is now about who benefits from the story. When a debate shifts from evidence to motive, it's a sign the evidence is no longer the primary focus. That is a jerk in the discourse, and it usually means the noise curve has decoupled from the signal curve.

The second is the unglamorous gap between permission and oversight. How much autonomy we give our agents to act on our behalf without anyone watching. We have not yet had the first well-documented case of an agent causing real damage on its own inside an ordinary firm rather than a lab. But with so many people handing agents access to email and to sensitive data, and automating tasks without review, that is the one I expect next. Hopefully, not to you or your business.

Two ways to hand over the keys

The risk in the last paragraph is not abstract, and it is yours to manage in the settings, not the AI labs. Here is what the responsible version looks like next to the risky one, line by line.

Where this leaves you

Right now the noise curve (the debate) and the signal curve (the AI capabilities) are decoupling, so the debate may not be tracking reality. The debate is accelerating.  The loudest version of the debate rests on a capability claim that cannot currently be measured, by the admission of the people whose job is to measure it.

I'd love to hand you an easy answer, but these are not black and white issues. Gebru is likely right that the doom frame is a distraction. The doomers could be right that the capability curve is steep. These are both important to know and be following. And neither of those should distract you from the thing you actually control, which is how much autonomy you have handed your own agents, and whether anyone is watching.

The five-minute practice

List the agents and automations your team uses, and what each one is connected to. (Look in Monday.com, Claude, ChatGPT, Salesforce, Zapier, Google Gems, etc. ) Then paste this into your AI tool of choice, and into a second one, and compare the two answers:

Here is a list of the AI agents and automations my team uses, and what each one is connected to: [paste your list].

For each one, tell me what could go wrong if it acted on its own or was manipulated, the single worst action it could take with the access it has, and the one change that would most reduce that risk. Rank the three riskiest.

Use at least two LLMs to ask this question of, because the disagreement between them is where the question you hadn't thought to ask tends to hide. If you are struggling to identify the list of agents and automations, that is clue that the cat is out of the bag but you should still be able to get it back in.

Best wishes,

Rose