burn-rate-alert-rule-generator: SLO burn-rate alerts for Prometheus without the hand-written PromQL

Release #2 in our open-source series. burn-rate-alert-rule-generator is a free tool that writes the alerts for a service's reliability goal: alerts that page people when a real problem is eating into the month's allowance for failure, and stay quiet for blips that fix themselves.

Why it matters

No service works perfectly all the time, so teams set a reliability goal, a service level objective (SLO): for example, "99.9% of checkout requests succeed over 30 days." The remaining 0.1% is the error budget, the amount of failure the team can absorb before the goal is broken. For a service that is completely down, 0.1% of a month is roughly 43 minutes.

Alerting on that goal is harder than it sounds:

• Too jumpy: an alert on every brief spike in errors wakes people at 3 a.m. for problems that fixed themselves.
• Too slow: a small, steady fault can use up a month's budget over a few weeks without ever looking dramatic enough to trip a simple alert.

The Google SRE Workbook's answer is burn-rate alerting. The burn rate is how fast the budget is being spent compared with a pace that would last exactly the month (1x). Instead of asking "are there errors right now?", an alert asks "are we spending the budget much faster than we can afford?" It checks a long window (such as the last hour) to show the problem is real, and a short window (such as the last five minutes) to show it is still happening.

The method is well known. The rules are tedious to write by hand: each SLO needs error rates measured over several time windows, four alerts with thresholds calculated from the goal, and labels that stay consistent across services. One wrong window or mistyped threshold and an alert never fires.

What it does

• Takes the goal as input: the objective (say 99.9% over 30 days) and the queries that count total and failed requests in Prometheus, a widely used monitoring system, written in its query language, PromQL.
• Does the arithmetic. For each goal it writes 11 rules: seven recording rules, which pre-compute the error rate over windows from five minutes to three days, and four alerts. Thresholds come from the objective, so changing 99.9% to 99.95% moves every alert at once.
• Separates urgent from slow. Two alerts are marked critical for fast burns (14.4x over an hour, 6x over six hours) and two warning for slow leaks (3x over a day, 1x over three days). Your existing alerting setup decides who gets paged and what becomes a ticket.
• States each alert in budget terms. A report shows how much of the month's budget has gone when each alert fires (2% for the fastest, 10% for the slowest), so engineers and managers can agree on the alerts before they are switched on.
• Changes nothing on its own. It writes a rule file and a report. Engineers review the file, check it with Prometheus's own validator, and load it through their normal change process.

Who it's for

• SRE and platform teams (site reliability engineering: the people who keep services up) who want every service's SLO alerts generated from one reviewed definition, not copied between rule files.
• Service owners who know their objective but not the PromQL that turns it into a sensible page.
• Anyone approving alerts, who wants to see what a page means in error budget terms first.

What it checks

The generator validates each SLO definition and checks the structure of its generated rules. From one SLO file, it produces the following for each SLO:

Output	What it is
Recording rules	slo:<service>_<slo>:error_ratio_<window> for 5m, 30m, 1h, 2h, 6h, 1d and 3d
Page alerts	14.4x burn over 1h and 5m, and 6x burn over 6h and 30m (severity critical by default)
Ticket alerts	3x burn over 1d and 2h, and 1x burn over 3d and 6h (severity warning by default)
Report	A Markdown or JSON table per SLO: threshold, severity, and budget spent per long window

The burn-rate pairs above are the Workbook defaults; each SLO can set its own. The error ratio comes from either errors / total or 1 - good / total, so availability and latency SLOs use the same file:

service: checkout-api
runbook_url: https://runbooks.example.com/checkout-api
labels:
  team: payments
slos:
  - name: availability
    description: Checkout API availability
    objective: 99.9
    window: 30d
    total: sum(rate(http_requests_total{service="checkout-api"}[{{window}}]))
    errors: sum(rate(http_requests_total{service="checkout-api", code=~"5.."}[{{window}}]))

{{window}} marks where each recording window goes.

Architecture and tech stack

Technology	Role	Why it was chosen
Python 3.11+	Runtime	Available on every CI runner and next to most Prometheus tooling
PyYAML	SLO input and rule output	Prometheus rule files are YAML, and so are most teams' SLO definitions
pytest	Tests	29 tests covering parsing, thresholds, budget maths, naming, and the CLI, with byte-for-byte golden files
promtool	External check	Prometheus's own validator confirms the generated file, not just our reading of it

PyYAML is the only runtime dependency. The tool makes no network requests and never talks to Prometheus or a pager; it writes a file you load yourself.

Key design decisions and trade-offs

• Optimized for reviewable output. Every alert carries summary, description and a budget_consumed_percent annotation, so whoever reads the page, or the pull request that adds it, sees what the threshold means.
• Optimized for one source of truth. Thresholds are computed from the objective, never typed. Change 99.9 to 99.95 and every threshold moves with it.
• Refuse rather than guess. An objective outside (0, 100), a short window that is not shorter than its long window, an expression without {{window}}, or two SLOs whose names collide after sanitizing all stop the run with exit code 2 before any file is written.
• Intentionally cut: deployment. The tool does not push rules to Prometheus or Alertmanager. Loading rules is a change your normal review and deploy process should own.
• Intentionally cut: for: durations. The paired windows already filter short spikes, so no for: is generated. Teams that want one add it themselves for now.

Under the hood

Each alert's threshold is the burn-rate factor times the error budget, and its annotation states how much of the whole SLO window's budget is spent during one long window at that rate. From generate.py:

def budget_consumed(pair: WindowPair, slo: SLO) -> float:
    """Share of the whole SLO window's error budget spent during the long window of a burn-rate alert that fires."""
    return pair.factor * parse_duration(pair.long) / parse_duration(slo.window)

# inside build_group, for each burn-rate pair
threshold = round(pair.factor * slo.error_budget, 10)
expr = (
    f"{record_name(slo, pair.long)} > {_fmt(threshold)} "
    f"and {record_name(slo, pair.short)} > {_fmt(threshold)}"
)

For a 99.9% objective the budget is 0.001, so the 14.4x page fires above an error ratio of 0.0144 on both the 1h and 5m recording rules. 14.4 × 1h ÷ 30d is 2% of the month's budget, which matches the Workbook's table. The report prints the same arithmetic for every SLO:

Kind	Long	Short	Factor	Threshold	Budget spent per long window
page	1h	5m	14.4x	0.0144	2%
page	6h	30m	6x	0.006	5%
ticket	1d	2h	3x	0.003	10%
ticket	3d	6h	1x	0.001	10%

Alerts only reference recording rules defined in the same group, and a structural check (valid metric and label names, no duplicates, no empty expressions) runs on the generated document before it is written.

Sharp edges and benchmarks

• Name collisions. Rule names are built from sanitized service and SLO names, so x-y and x_y in one file would produce the same rules. The tool refuses that file with a duplicate rule name error instead of silently overwriting one SLO's rules with another's. Renaming one SLO is the fix.
• PromQL is not parsed. The tool checks that {{window}} is present but not that the expression is valid PromQL. A typo passes the generator and is caught by promtool, which is why the README tells you to run it.
• Validation, honestly scoped. The demo SLO file is synthetic. The generated file passes promtool check rules (SUCCESS: 22 rules found) from the official prom/prometheus:v3.5.0 image, which checks syntax and references. It has not been evaluated against real traffic, and no benchmarks are claimed: generation is a few dozen rules from a small YAML file.

Quick start

pip install "git+https://github.com/jare20895/burn-rate-alert-rule-generator"
git clone https://github.com/jare20895/burn-rate-alert-rule-generator
cd burn-rate-alert-rule-generator
burn-rate-alert-rule-generator examples/slos.yaml --rules-out rules.yaml

That writes 22 rules in two groups (an availability SLO and a latency SLO) and prints the budget report. Check the output with Prometheus's validator, here from Docker:

docker run --rm --entrypoint sh -v "$PWD:/data:ro" prom/prometheus:v3.5.0 \
    -c 'promtool check rules /data/rules.yaml'

Exit codes are CI-friendly: 0 when rules are generated with no findings, 1 when there are warnings (no runbook URL, or a long window longer than the SLO window), and 2 when the SLO file is invalid.

Path to production

• Optional for: per burn-rate pair, defaulting to the Workbook's suggested values, for teams that want extra damping.
• Light PromQL checks, enough to confirm that {{window}} sits inside a range selector.
• Calendar SLO windows and one rule file per SLO for teams that own their SLOs separately.

Limits, plainly

It generates rules; it does not choose your objectives or windows, deploy anything, or prove the alerts suit your traffic. Rolling SLO windows only. No for: is generated. The budget figures assume a rolling window.

The code is MIT licensed: github.com/jare20895/burn-rate-alert-rule-generator.