Escalation Policies

Escalation policies define how alerts are routed when incidents occur. They ensure that if the first responder doesn't acknowledge an incident, it automatically escalates to additional team members until someone responds.

Why Use Escalation Policies?

Without escalation:

• Alerts go to one person who might be unavailable
• Critical incidents go unnoticed
• Response times suffer

With escalation policies:

• Multiple team members are notified in sequence
• Unacknowledged incidents escalate automatically
• Someone always responds to critical issues

How Escalation Works

When a monitor triggers an incident:

1. Step 1 fires immediately — Primary responders notified
2. If not acknowledged within the delay, Step 2 fires — Secondary responders notified
3. Pattern continues through all steps until acknowledged

Once someone acknowledges the incident, escalation stops.

Creating an Escalation Policy

Escalation Steps

Steps are the building blocks of escalation policies.

Step Configuration

Example: 3-Tier Escalation

Step 1 (0 min delay):
  - Notify: Slack #alerts channel
  - Notify: On-call engineer

Step 2 (5 min delay):
  - Notify: Email to engineering team
  - Notify: SMS to team lead

Step 3 (15 min delay):
  - Notify: Voice call to manager
  - Notify: Slack #emergency channel

Assigning Policies to Monitors

Link escalation policies to monitors so incidents trigger the correct escalation flow.

Step Timing

Understanding how delays work:

Timeline Example

For a policy with steps at 0, 5, and 15 minutes:

00:00 - Incident created
00:00 - Step 1 fires (0 min delay)
00:05 - Step 2 fires if not acknowledged (5 min delay)
00:15 - Step 3 fires if not acknowledged (15 min delay)

Choosing Delays

Integrating with On-Call Schedules

Each step can optionally notify the current on-call person from an on-call schedule.

Enable On-Call Notification

When creating a step:

1. Enable Notify On-Call
2. The step will notify whoever is currently on-call for the team

How It Works

1. Step fires based on its delay
2. PixoMonitor looks up current on-call person from the schedule
3. Notifications sent to that person's configured channels
4. If on-call has changed during escalation, the new person is notified

Team-Based Policies

Link escalation policies to teams for organized team management.

Create a Team Policy

Team policies help organize escalations by department or function (e.g., "Database Team", "Frontend Team").

Editing Policies

Update existing policies as your team evolves.

Update Policy Details

1. Go to Settings → Escalation Policies
2. Click on the policy to edit
3. Update name, description, or team assignment
4. Click Save

Update Steps

When updating a policy, you can:

• Add new steps
• Remove existing steps
• Change step order
• Modify delay times
• Update alert channels

Deleting Policies

Remove policies that are no longer needed.

1. Go to Settings → Escalation Policies
2. Find the policy to delete
3. Click Delete

Before deleting, consider:

• Which monitors use this policy?
• Should they be assigned to a different policy first?

Example Policies

Small Team (2-3 people)

Step 1 (0 min):
  - Slack #alerts
  - On-call person

Step 2 (10 min):
  - SMS to all team members

Medium Team (5-10 people)

Step 1 (0 min):
  - Slack #alerts
  - On-call person

Step 2 (5 min):
  - Email to team
  - Secondary on-call

Step 3 (15 min):
  - SMS to team lead
  - Voice call to on-call

Large Team / Enterprise

Step 1 (0 min):
  - PagerDuty integration (webhook)
  - On-call engineer

Step 2 (5 min):
  - Secondary on-call
  - Slack #incidents

Step 3 (10 min):
  - Team lead (SMS + Voice)
  - Slack #engineering-urgent

Step 4 (20 min):
  - Engineering manager
  - Executive Slack channel

Escalation and Acknowledgement

What Stops Escalation

Escalation stops when:

• Someone acknowledges the incident
• The incident is resolved
• All steps have fired

What Doesn't Stop Escalation

Escalation continues even if:

• Someone views the incident without acknowledging
• Alerts are sent (sending doesn't equal acknowledging)

Best Practices

1. Start broad, then narrow — First step to a channel, later steps to individuals
2. Use increasing urgency — Slack → Email → SMS → Voice
3. Include on-call — Always notify on-call in early steps
4. Keep delays reasonable — Balance quick response with alert fatigue
5. Test your policies — Verify escalation works before relying on it
6. Document policies — Make sure team knows what to expect
7. Review regularly — Update as team composition changes