· 9 min read · Incident Management

When to Take Humans Off On-Call: The Case for Autonomous Incident Remediation

70% of SREs cite on-call stress as a direct contributor to burnout. The standard response — better rotation, smarter paging, on-call management tooling — addresses the symptom. The cause is that humans are being woken up for incidents that do not require a human. Here is the case for removing them from that loop entirely.


The Catchpoint SRE Report 2025 found that nearly 70% of SREs identify on-call stress as a direct contributor to burnout. A separate StackGen survey found that teams receiving more than 10 pages per week see significantly higher attrition — engineers leave, and the remaining engineers absorb the load, which drives more pages, which drives more attrition. The cycle is self-reinforcing.

The industry's response to this data has been a wave of on-call management tooling: smarter escalation policies, better noise reduction, fairer rotation scheduling, quieter paging windows. PagerDuty, Rootly, Grafana OnCall, and others compete for this market. These tools are genuinely useful. They are also fundamentally misdiagnosed as solutions to burnout.

On-call burnout is not primarily caused by bad scheduling or noisy alerts. It is caused by being woken at 3am for incidents that follow a known pattern, have a known fix, and do not require human judgment. The engineer is in the loop not because their expertise is needed, but because the system was not built to handle known-pattern incidents without them.

That is an automation problem, not a scheduling problem.

70%
of SREs cite on-call stress as a burnout driver
Catchpoint SRE Report 2025
30%
rise in operational toil in 2025
Runframe 2026
~3,000
avg daily alerts per enterprise org
Vectra AI 2026
40%
of alerts are never investigated
Vectra AI 2026

Why the standard fixes do not work

The tooling industry's response to on-call burnout has followed a consistent pattern: reduce the noise that reaches engineers, distribute the load more fairly, and make the paging experience less disruptive when it happens. These are real improvements. They do not address the root cause.

Alert noise reduction

ML-based grouping and deduplication reduce the volume of alerts that reach engineers. This is meaningful — fewer pages means less disruption. But it does not eliminate the page for known-pattern incidents. It filters the queue; it does not resolve what is in the queue.

Fairer rotation and scheduling

Distributing on-call load more evenly reduces the burden per engineer. It does not reduce the total burden. If the same 3am pages are going to 6 engineers instead of 2, the aggregate disruption to the team is unchanged — it is just spread more broadly.

Better runbooks and escalation paths

Well-written runbooks reduce resolution time once an engineer is paged. They do not prevent the engineer from being paged in the first place. The incident still interrupts a person's sleep; it just takes 8 minutes to resolve instead of 25.

The common thread: all of these interventions assume a human must be in the loop. They optimize the experience of being on-call rather than questioning which incidents require on-call at all.

Which incidents do not need a human

Not all incidents are equal. A production database corruption affecting multiple customers requires judgment, experience, and careful decision-making — exactly the expertise an experienced engineer brings at 3am. A disk filling up on a non-critical host, a service restart after a known OOM pattern, a cache flush triggered by a predictable load spike — these do not.

The incidents that are driving burnout are not the complex, novel ones. Engineers do not resent being woken for genuine crises. They resent being woken for incidents that follow a pattern they have resolved manually dozens of times.

Incidents that are candidates for autonomous remediation

·Service restarts on known OOM patterns
·Disk cleanup on non-critical hosts
·Cache invalidation and flush operations
·Auto-scaling responses to load spikes
·Certificate renewal for non-production environments
·Queue backlog clearing via known consumer restart
·Failed health check recovery via pod reschedule
·DNS cache flush for known resolution failures
·Log rotation and cleanup on known-full volumes
·Temporary network partition recovery via failover

These are pattern-matched, known-outcome incidents. They have runbooks. An engineer follows the same steps every time. They are automation waiting to be built.

The human-in-the-loop threshold

Autonomous remediation does not mean removing humans from all incidents. It means removing humans from incidents where their judgment is not required — and ensuring human approval is mandatory for incidents where it is.

The threshold is defined by risk, not complexity. An incident that has a known fix, affects a non-critical system, and carries low blast radius if the remediation goes wrong is a candidate for autonomous execution. An incident affecting a production database, a customer-facing payment flow, or a system with compliance implications should require human approval before any action is taken — regardless of how well understood the pattern is.

Autonomously resolvable

  • · Known pattern with documented runbook
  • · Non-critical system or non-customer-facing service
  • · Low blast radius if remediation fails
  • · Reversible action (restart, flush, scale-up)
  • · No compliance or audit implications

Requires human approval

  • · Production database modifications
  • · Customer-facing payment or auth systems
  • · Irreversible actions (data deletion, schema changes)
  • · Novel pattern not in runbook library
  • · High blast radius or compliance scope

The practical implementation is a risk scoring system. Before any automated action, the system scores the incident against a configurable risk matrix — affected systems, blast radius, action reversibility, compliance scope. Below the threshold, the runbook executes. Above it, the system routes for human approval instead of paging someone to execute a known fix manually.

What it looks like in practice

An autonomous remediation pipeline covers the full incident lifecycle without requiring a human until risk thresholds demand it:

1

Detection

An alert fires from your monitoring stack — Splunk, Dynatrace, Grafana, or any other source. The pipeline receives the signal.

2

Signal qualification

An AI layer classifies the alert: is this a genuine incident or a noise event? What is the likely category? What runbooks are associated with this pattern?

3

CMDB enrichment

The incident is enriched with live infrastructure context — which services are affected, what their dependencies are, who owns them, and what the blast radius looks like.

4

Risk scoring

The enriched incident is scored against a configurable risk matrix. Below threshold: proceed to execution. Above threshold: route for human approval.

5

Runbook execution

The appropriate runbook executes with deterministic, auditable steps. No LLM in the execution path — AI handles judgment, pipelines handle action.

6

Verification

The fix is verified: did the service recover? Did the metric return to baseline? If not, escalate to human.

7

Ticket closure

A work note is generated documenting what happened, what was done, and what the outcome was. The ticket closes.

The engineer is paged only when the risk score is above threshold or when verification fails after remediation. Everything else resolves without waking anyone up.

The objection worth taking seriously

The most common objection to autonomous remediation is trust: how do you know the system will make the right call? It is a legitimate question. The answer is not "trust the AI" — it is "design the system so that trust is not required for high-risk actions."

Autonomous execution should only occur below a configurable risk threshold — for incidents where the blast radius of a wrong call is bounded and recoverable. Above that threshold, the system stops and routes for human approval. The AI is not replacing engineer judgment for complex, high-stakes decisions. It is replacing the manual execution of known-pattern fixes that an engineer would execute the same way every time anyway.

The audit trail is also non-negotiable. Every autonomous action — what was detected, what was scored, what runbook was selected, what commands were executed, what the outcome was — needs to be logged with full provenance. When something goes wrong (and occasionally it will), the log is what lets you understand what happened and improve the threshold.

Axiometica AIR is built around this model — autonomous execution below configurable risk thresholds, mandatory human approval above them, and a full audit trail of every action taken.

Self-hosted. Local LLM support. Docker Compose. Free for internal use.