# Prometheus Alertmanager alerts on your phone: severity routing, grouping and quiet resolves

> Send Prometheus Alertmanager alerts to iPhone and Android as push notifications. One push per alert group, critical alerts loud, warnings normal, resolved notifications quiet. A tested alertmanager.yml, example rules, and when the Alertmanager 0.32 payload option is the better choice.

*29.10.2026*

<!-- source: https://pocketalert.app/blog/prometheus-alertmanager-notifications -->

Prometheus is good at noticing that something is wrong. Getting that news to a person is Alertmanager's job, and in many setups the news ends up in an email folder or a Slack channel that nobody reads at 3 a.m.

Alertmanager has no push app of its own, but it has a generic webhook receiver, and that's all you need to get alerts on your phone. This guide sets it up with severity routing: `critical` alerts arrive loud, warnings arrive as normal notifications, and "resolved" arrives quietly. Everything here was tested with Alertmanager 0.34.2 and Prometheus 3.15.1 in Docker. The payloads were captured and run through Pocket Alert's own template code.

## What you need

- Alertmanager 0.31 or newer for the full setup. Older versions are covered below.
- The Pocket Alert app on your [iPhone or Android](/get-app) phone, and an API key from [API Keys](/account/apikeys).
- A paid plan for the three-webhook setup, because the Free plan includes one webhook. A one-webhook version is near the end.

## What Alertmanager sends

Alertmanager doesn't send one request per alert. It collects alerts into groups by the labels in `group_by`, and sends one notification per group: a POST with a fixed JSON body, `"version": "4"`. When two hosts go down under `group_by: ['alertname']`, one request goes out. Here it is, trimmed a little:

```json
{
  "receiver": "pocketalert-critical",
  "status": "firing",
  "alerts": [
    {
      "status": "firing",
      "labels": {"alertname": "InstanceDown", "instance": "web-1:9100", "job": "node", "severity": "critical"},
      "annotations": {"summary": "web-1:9100 is down"},
      "startsAt": "2026-10-31T16:17:04.374Z",
      "generatorURL": "http://prometheus.example.com:9090/graph?g0.expr=up%7Bjob%3D%22node%22%7D+%3D%3D+0&g0.tab=1"
    },
    {
      "status": "firing",
      "labels": {"alertname": "InstanceDown", "instance": "web-2:9100", "job": "node", "severity": "critical"},
      "annotations": {"summary": "web-2:9100 is down"}
    }
  ],
  "notification_reason": "first notification",
  "groupLabels": {"alertname": "InstanceDown"},
  "commonLabels": {"alertname": "InstanceDown", "job": "node", "severity": "critical"},
  "commonAnnotations": {},
  "routeLabels": {},
  "externalURL": "http://alertmanager.example.com:9093",
  "version": "4",
  "groupKey": "{}/{severity=\"critical\"}:{alertname=\"InstanceDown\"}",
  "truncatedAlerts": 0
}
```

Four things in this payload shape the setup:

- **One push per group.** The individual alerts are in the `alerts` array, so the message template has to work with an array.
- **`commonAnnotations` is often empty.** It only holds annotations that are identical on every alert in the group. A summary that mentions the instance is different on every alert, so it never shows up there.
- **`status` belongs to the group.** It is `resolved` only when every alert in the group is resolved. When one host recovers and another is still down, the status stays `firing` and the recovered alert is listed inside with its own `"status": "resolved"`.
- **There is no top-level `level` field**, and that's where a Pocket Alert webhook looks for a priority. So the priority comes from the webhook's default level, which is why this guide uses one webhook per loudness.

## Step 1: Create the Pocket Alert webhooks

<div class="cmp-table-wrap">
<table class="cmp-table">
<thead><tr><th>Webhook</th><th><code>default_level</code></th><th>Receives</th></tr></thead>
<tbody>
<tr><th>Prometheus critical</th><td><code>2</code> (critical)</td><td>Firing groups with <code>severity="critical"</code></td></tr>
<tr><th>Prometheus warning</th><td><code>0</code> (default)</td><td>All other firing groups</td></tr>
<tr><th>Prometheus resolved</th><td><code>-1</code> (low)</td><td>Groups where every alert has resolved</td></tr>
</tbody>
</table>
</div>

Buttons can only be set through the API, so create all three with one script:

```bash
PA_KEY='your-api-key'

FIRING='%groupLabels.alertname%: %alerts.#(status=="firing")#|#% firing
%alerts.#(status=="firing").annotations.summary%
Firing: %alerts.#(status=="firing")#.labels.instance%'

RESOLVED='Resolved: %groupLabels.alertname% (%commonLabels.severity%)
%alerts.#.labels.instance%'

BUTTONS='[{"type":"view","label":"Alertmanager","value":"%externalURL%"}]'

create_webhook() {
  jq -n --arg name "$1" --argjson level "$2" --arg message "$3" --arg actions "$4" \
    '{name: $name, default_level: $level, message: $message, actions: $actions}' |
  curl -sS https://api.pocketalert.app/v1/webhooks \
    -H "Token: $PA_KEY" \
    -H 'Content-Type: application/json' \
    --data-binary @- |
  jq -r '"\(.name): https://p4a.me/wh/\(.url)"'
}

create_webhook 'Prometheus critical' 2 "$FIRING" "$BUTTONS"
create_webhook 'Prometheus warning' 0 "$FIRING" "$BUTTONS"
create_webhook 'Prometheus resolved' -1 "$RESOLVED" ''
```

It prints three receive URLs. Keep the slug at the end of each one; you'll paste the slugs into `alertmanager.yml`.

The `%...%` placeholders are [GJSON paths](https://info.pocketalert.app/api/webhooks/settings), and GJSON can query arrays:

- `alerts.#(status=="firing")#|#` counts the alerts in the group that are still firing.
- `alerts.#(status=="firing").annotations.summary` is the summary of the first firing alert.
- `alerts.#(status=="firing")#.labels.instance` lists the instances of all firing alerts. It renders as a JSON array, like `["web-1:9100","web-2:9100"]`, because there is no string-join function.

A path that doesn't exist renders as an empty string, so a bad path gives you a blank line, not a lost alert. Pocket Alert titles every webhook push `Webhook <name>`, so the title reads "Webhook Prometheus critical" and the alert name goes on the first line of the message. `critical` needs a paid plan, which this setup needs anyway for the three webhooks. It plays through the silent switch and Do Not Disturb on iPhone and takes over the screen on Android. Use `1` (high) instead if you don't want alerts to override Do Not Disturb.

## Step 2: alertmanager.yml

Replace the three `*_SLUG` placeholders with your slugs:

```yaml
route:
  receiver: pocketalert-warning
  group_by: ['alertname']
  group_wait: 30s
  group_interval: 5m
  repeat_interval: 4h
  routes:
    - matchers: ['severity="critical"']
      receiver: pocketalert-critical
      group_wait: 10s
      repeat_interval: 1h
    - matchers: ['severity="info"']
      receiver: blackhole

receivers:
  - name: pocketalert-critical
    webhook_configs:
      - url: 'https://p4a.me/wh/{{ if eq .Status "resolved" }}RESOLVED_SLUG{{ else }}CRITICAL_SLUG{{ end }}'
        send_resolved: true
        max_alerts: 20
  - name: pocketalert-warning
    webhook_configs:
      - url: 'https://p4a.me/wh/{{ if eq .Status "resolved" }}RESOLVED_SLUG{{ else }}WARNING_SLUG{{ end }}'
        send_resolved: true
        max_alerts: 20
  - name: blackhole

inhibit_rules:
  - source_matchers: ['severity="critical"']
    target_matchers: ['severity="warning"']
    equal: ['alertname', 'instance', 'mountpoint']
```

How the pieces work:

- **Routing by severity.** `severity="critical"` goes to the loud receiver, `info` is dropped, and everything else, including alerts without a `severity` label, falls through to the warning receiver.
- **The URL picks the webhook.** Since 0.31.0 the webhook `url` is a template. A group whose status is `resolved` goes to the quiet webhook, and everything else goes to the receiver's own webhook. `send_resolved` is already `true` by default for webhooks; it's written out so nobody switches it off without noticing.
- **`max_alerts: 20`** limits how many alerts go into one request. In the test, 25 hosts went down at once. That produced one push reading "20 firing", and `truncatedAlerts: 5` in the payload.
- **Inhibition.** When the critical `DiskSpaceLow` fires for a filesystem, the warning for the same `alertname`, `instance` and `mountpoint` is muted. `amtool alert query --inhibited` shows it as `suppressed`.

Check the config and the routing before reloading:

```bash
amtool check-config alertmanager.yml
amtool config routes test --config.file=alertmanager.yml severity=critical alertname=Test
```

The second command prints `pocketalert-critical`.

Start Alertmanager with `--web.external-url` set to the address you can open from your phone. Without it, `externalURL` is the bare hostname. In Docker that was `http://de48ea5ffc29:9093`, which makes for a useless button.

### Check your version first

`alertmanager --version` has to say 0.31 or newer for the templated URL. Version 0.30.1 accepts this exact config, and `amtool check-config` reports SUCCESS, but it doesn't render the template. It sent the request to `/wh/%7B%7B%20if%20eq%20.Status...`, which matches no webhook. On older versions, give each receiver the plain URL of its own webhook. Resolved notifications then arrive at that webhook's level.

## Step 3: Rules with a severity label

The routing above only works if your rules set `severity`. Here are two examples (more ideas on what to watch are on the [server monitoring](/use-cases/server-monitoring) page), with `DiskSpaceLow` at two thresholds so the inhibit rule has something to do:

```yaml
groups:
  - name: node
    rules:
      - alert: InstanceDown
        expr: up{job="node"} == 0
        for: 2m
        labels:
          severity: critical
        annotations:
          summary: '{{ $labels.instance }} is down'

      - alert: DiskSpaceLow
        expr: node_filesystem_avail_bytes{fstype!~"tmpfs|overlay"} / node_filesystem_size_bytes * 100 < 15
        for: 10m
        labels:
          severity: warning
        annotations:
          summary: '{{ $labels.instance }} {{ $labels.mountpoint }} has {{ printf "%.0f" $value }}% free'

      - alert: DiskSpaceLow
        expr: node_filesystem_avail_bytes{fstype!~"tmpfs|overlay"} / node_filesystem_size_bytes * 100 < 5
        for: 5m
        labels:
          severity: critical
        annotations:
          summary: '{{ $labels.instance }} {{ $labels.mountpoint }} has {{ printf "%.0f" $value }}% free'
```

`promtool check rules rules.yml` validates the file. Point Prometheus at the file and at Alertmanager in `prometheus.yml`:

```yaml
rule_files: [rules.yml]
alerting:
  alertmanagers:
    - static_configs:
        - targets: ['alertmanager:9093']
```

## Test it with amtool

Fire a test alert, wait for `group_wait` (10 seconds on the critical route), and resolve it:

```bash
amtool --alertmanager.url=http://localhost:9093 alert add alertname=PocketAlertTest severity=critical instance=test:9100 \
  --annotation='summary="Test alert from amtool"'

amtool --alertmanager.url=http://localhost:9093 alert add alertname=PocketAlertTest severity=critical instance=test:9100 \
  --annotation='summary="Test alert from amtool"' --end="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
```

Keep the inner double quotes in the annotation, or amtool warns that it fell back to its old parser. The resolved push arrives at the next `group_interval` tick, which can take up to 5 minutes with this config.

## What lands on your phone

These are the messages Pocket Alert renders from the real payloads in the test, where Prometheus saw `web-1` and `web-2` go down and come back:

```text
Webhook Prometheus critical
InstanceDown: 2 firing
web-1:9100 is down
Firing: ["web-1:9100","web-2:9100"]
```

When `web-1` recovers, Alertmanager sends an update at the next `group_interval`. The group is still firing, so it goes to the critical webhook:

```text
Webhook Prometheus critical
InstanceDown: 1 firing
web-2:9100 is down
Firing: ["web-2:9100"]
```

When both are back, a low-priority push arrives:

```text
Webhook Prometheus resolved
Resolved: InstanceDown (critical)
["web-1:9100","web-2:9100"]
```

## Timers: avoiding push storms

<div class="cmp-table-wrap">
<table class="cmp-table">
<thead><tr><th>Setting</th><th>What it does</th><th>Here</th></tr></thead>
<tbody>
<tr><th><code>group_wait</code></th><td>How long a new group waits for more alerts before the first push</td><td>30s, 10s for critical</td></tr>
<tr><th><code>group_interval</code></th><td>Minimum gap between pushes for a group whose alerts changed</td><td>5m</td></tr>
<tr><th><code>repeat_interval</code></th><td>Reminder for a group that is still firing and unchanged</td><td>4h, 1h for critical</td></tr>
</tbody>
</table>
</div>

A group can push at most once per `group_interval`, so a flapping alert costs up to 12 pushes an hour at 5m. An outage that lasts a day costs 24 reminders at a 1h `repeat_interval`. Grouping does most of the work: a switch failure that takes down 40 hosts is one push, not 40. Don't use `group_by: ['...']`, which groups by every label and turns that back into one push per alert. Plans have [daily message limits](https://info.pocketalert.app/features/daily-limits) (Starter is 1,000 a day), and every request to a webhook that produces a message counts against it.

## Buttons: Alertmanager and Grafana

The `BUTTONS` template above adds an "Alertmanager" button. You can add up to three [action buttons](/features/action-buttons). This one opens a Grafana dashboard for the first firing instance:

```bash
BUTTONS='[{"type":"view","label":"Alertmanager","value":"%externalURL%"},{"type":"view","label":"Grafana","value":"https://grafana.example.com/d/YOUR_DASHBOARD?var-instance=%alerts.#(status=="firing").labels.instance%"}]'
```

The unescaped quotes inside the placeholder look wrong, but they work: Pocket Alert substitutes placeholders first and parses the JSON afterwards.

Watch out for empty values. A button whose `value` comes out empty is dropped from that push; the other buttons and the message still arrive. `%alerts.0.generatorURL%` (the Prometheus graph link) is a good target for alerts from Prometheus, but alerts sent with amtool have no generator URL, so they arrive without that button. Action buttons are a paid feature.

## Alertmanager 0.32+: calling the API directly

Older guides say Alertmanager can't change the webhook body. That changed in 0.32.0, which added a `payload` option to `webhook_configs`. Together with `http_config.http_headers`, which can send a `Token` header, it means Alertmanager can POST straight to `https://api.pocketalert.app/v1/messages` with its own title and level.

There's a trap. Alertmanager parses every rendered value as YAML. In the test, a field-by-field payload with `message: '{{ range .Alerts }}{{ .Annotations.summary }}...'` turned the summary `web-1:9100: / has 12% free` into the object `{"web-1:9100": "/ has 12% free"}`. The API rejects that with `400`. Alertmanager doesn't retry a `4xx` immediately; it sends the same broken body again at every `group_interval`, so the alert never arrives.

Building the whole body with `dict` and `toJson` avoids it:

```yaml
receivers:
  - name: pocketalert-api
    webhook_configs:
      - url: https://api.pocketalert.app/v1/messages
        http_config:
          http_headers:
            Token:
              files: [/etc/alertmanager/pocketalert.key]
        payload: >-
          {{ $msg := "" }}{{ range .Alerts }}{{ $msg = printf "%s%s: %s\n" $msg .Status .Annotations.summary }}{{ end }}
          {{ $level := "default" }}{{ if eq .Status "resolved" }}{{ $level = "low" }}{{ else if eq .CommonLabels.severity "critical" }}{{ $level = "critical" }}{{ end }}
          {{ dict "title" (printf "[%s] %s" (toUpper .Status) .GroupLabels.alertname) "message" $msg "level" $level | toJson }}
```

With a summary containing a colon and quotes, it sent:

```json
{"level":"critical","message":"firing: web-1:9100: / has 4% free, \"quoted\"\nfiring: web-2:9100 / has 3% free\n","title":"[FIRING] DiskSpaceLow"}
```

The resolved version came through with `"level":"low"`.

So which should you use? The direct API gives you a real title and a level for each notification, all from one receiver and with no webhooks. The webhook needs no API key in Alertmanager at all and works on any version (the plain-URL variant). A bad path renders as a blank line instead of a rejected request, and every request shows up in [webhook history](/features/webhook-history). Alertmanager's own docs label `payload` "use at your own risk", so test it with amtool before you rely on it.

## On the Free plan

The Free plan has one webhook, 50 messages a day, no buttons, and `critical` is delivered as `high`. Create one webhook with `default_level: 1`, put its plain URL in a single receiver, and drop the severity routes. Resolved notifications then arrive at `high` too, unless you set `send_resolved: false`. On Alertmanager 0.32+, the direct API config above is the better fit for Free, because it needs no webhook and sets the level for each notification.

## If nothing arrives

- **Read the Alertmanager log.** A failed delivery logs `Notify for alerts failed` with the status code. A `404` means a wrong slug, or a templated URL on a version older than 0.31.
- **Open the webhook history** in Pocket Alert. If the request is there, the problem is the template. If it isn't, Alertmanager never sent it.
- **Blank lines** mean a path doesn't match the payload. Field names are case-sensitive.
- **A `%` right before a placeholder needs doubling.** A `%` followed by a space is plain text, so `CPU over 90% on %groupLabels.alertname%` works. Write `%%` when the `%` touches other text, as in `%commonLabels.usage%%%`.
- **Use the [webhook tester](/tools/webhook)** to capture what your Alertmanager actually sends before you write templates.

## FAQ

### Does this work with kube-prometheus-stack?

Yes. The receiver is ordinary Alertmanager configuration, so it goes into the chart's Alertmanager config like any other receiver. Check which Alertmanager version the chart deploys before you use the templated URL or `payload`. See [Kubernetes alerts](/use-cases/kubernetes-alerts) for the cluster side.

### I use Grafana alerting, not Alertmanager. What changes?

Grafana sends its own webhook format. See [the Grafana guide](/blog/how-to-integrate-grafana-into-pocketalert).

[Create a free account](/signup), run the script, paste three slugs into `alertmanager.yml`, and the next `InstanceDown` will reach your phone.