Most engineering organizations measure incident outcomes MTTR, customer impact, SLA compliance. Fewer measure the on-call process that produces those outcomes. This is a significant blind spot. Improving incident response without measuring on-call performance is like improving a manufacturing line without measuring the production process you can see whether the end product is better, but you cannot identify which changes produced the improvement or which process steps are creating constraints.
On-call performance measurement is a discipline that turns anecdotal observations (“it feels like we have been getting paged more lately”) into actionable data (“P1 MTTA increased 40% over the last six weeks for the payments service, and the primary escalation is firing on 30% of incidents”).
The Metrics That Matter
Mean Time to Acknowledge (MTTA) by priority and time window. MTTA is the most direct measure of on-call process effectiveness. Track it separately for P1 and P2 incidents, and separately for business hours versus off-hours. Aggregate MTTA hides the off-hours performance gaps that create the most customer impact. A team with 2-minute average MTTA during business hours and 25-minute average MTTA at 3 AM has a serious on-call infrastructure problem obscured by a flattering average.
Escalation rate. The percentage of incidents where the primary responder does not acknowledge within the escalation window and the system escalates to the secondary. A consistently high escalation rate indicates either that the primary is unreachable or that the notification channels are not working effectively. A low escalation rate indicates that the on-call structure is working or that the escalation window is too long to catch real gaps.
Alert volume per engineer per shift. Track how many pages each on-call engineer receives per shift over time. This metric reveals rotation fairness issues and alert noise trends simultaneously. If one engineer consistently receives significantly more pages than others on the same rotation, there is either a service ownership imbalance or a routing configuration problem.
False positive rate. The percentage of incidents that are acknowledged and then resolved without any documented action. These are the incidents that did not need to be incidents they represent alert noise that passed through correlation and reached the on-call engineer anyway. Tracking false positive rates over time reveals whether alert quality is improving or degrading.
Incident duration by priority and service. MTTR by priority and service identifies which services are hardest to restore and which priorities are systematically under-resourced. Services with consistently long resolution times are candidates for runbook improvement, additional documentation, or architectural review.
How to Collect These Metrics
Effective on-call performance measurement requires incident management software that captures the relevant timestamps and metadata automatically. Manual tracking is unreliable at scale and introduces measurement errors that undermine the analysis.
The minimum data requirements are: incident creation timestamp, alert source and severity, primary responder assignment, acknowledgment timestamp, escalation timestamps if applicable, and resolution timestamp. From these, MTTA, escalation rate, and incident duration are calculable. Alert volume per engineer and false positive rate require additional metadata responder identity and resolution classification.
ITOC360 captures all of this metadata automatically as part of the incident lifecycle. The on-call product includes reporting capabilities that surface MTTA trends, escalation rates, and rotation load distribution without requiring custom analytics work.
How to Use the Metrics
Metrics without action cycles are vanity. The value of on-call performance measurement is in the decisions it drives.
Weekly team reviews. Review MTTA trends and escalation rates in team operations meetings. When numbers degrade, investigate the specific incidents that drove the degradation rather than the aggregates. Aggregates identify that something changed. Individual incidents reveal what changed and why.
Monthly rotation reviews. Review alert volume distribution across engineers. Address imbalances through rotation restructuring, service ownership rebalignment, or alert threshold review. Make the data visible to the team engineers who can see their own on-call load relative to teammates will surface imbalance concerns before they reach the attrition stage.
Quarterly infrastructure reviews. Review false positive rates and escalation rates at the service level. Services with high false positive rates need alert threshold review. Services with high escalation rates need notification channel review or schedule restructuring.
Incident postmortems. Every significant incident should include an analysis of the on-call response: how long did acknowledgment take, did escalation fire, what context was available at acknowledgment, what slowed the response. This micro-level analysis drives the specific improvements that aggregate metrics identify as necessary.
On-call performance measurement is not punitive it is diagnostic. The goal is not to assess individual engineer performance but to identify where the incident management system is creating friction and where process improvements will have the most impact. Teams that measure on-call performance systematically improve faster than teams that rely on intuition. The data makes the path forward obvious.