Building alerts that respect attention

The fastest way to make a monitoring tool useless is to make it loud. Once alerts become noise, people mute them, and then the one that mattered goes unread. So before writing any email-sending code, we wrote down one rule: alert on transitions, not states.

Transitions, not states

A monitor that's down stays down until it isn't. That's one event when it fails and one when it recovers, not a reminder every check interval. Internally, the engine compares each check against the previous state and only acts on the change: up to down, up to degraded, down to up. Everything else is silence.

Degraded gets the same treatment. Crossing your response-time threshold is a transition worth knowing about; hovering above it for an hour is not forty more.

Recovery is part of the story

A down alert without a recovery alert leaves everyone refreshing the dashboard. Recovery emails close the loop, and they carry the outage duration, so the incident report writes its own first line.

The right inbox

By default, alerts go to every workspace member, the safe choice for small teams. But growing teams route differently, so recipients can be switched to a custom list: an on-call address, a client's inbox, a shared channel's email gateway. Slack and webhooks are next on this path.