Feature Flag Rollout Checklist
A feature flag rollout checklist for progressive and canary releases: define success metrics, roll out in stages, set a kill switch and clean up the flag after.
- Typical schedule per feature flag
- The checklist 5 sections, 18 items
- Variables
{{flag_name}}{{feature}} - Start from CI
Use this checklist when you release a feature behind a feature flag and turn it on step by step: internal users first, then a small group of customers, then everyone. It works with any flag tool, such as LaunchDarkly, Unleash, Flagsmith or a simple flag table in your own database. The run covers the whole life of the flag, from the first rollout step to the day the flag is removed from the code.
Who owns it
The engineer or product manager responsible for the feature owns the run. They decide when to move to the next stage, based on the agreed metrics. Someone on call must know the flag exists and how to turn it off.
Common pitfalls
- No success criteria. Without agreed metrics, every stage ends with "looks fine". Decide before the rollout what you will watch and what counts as a problem.
- The off state was never tested. Turning a flag off should be safe, but data written by the new code path can break the old one. Test switching off as well as on.
- Unknown behaviour when the flag service fails. Check what your code does if the flag cannot be evaluated. The default should be the safe, old behaviour.
- Flags that never die. Old flags pile up, make code hard to read and create untested combinations. Put the cleanup in the checklist and give it a date.
- Rolling out by gut feeling. Wait long enough at each stage to see real usage, including a busy period.
The checklist: Roll out flag {{flag_name}}
Prepare
-
Flag
{{flag_name}}for{{feature}}is documented with its owner and purpose -
Success metrics and rollback signals are written down
Examples: error rate on the new endpoint, conversion on the changed page, support tickets about the feature.
-
Default value is off, and the code falls back to the old behaviour if the flag service fails
-
Both flag states are covered by automated tests
-
Turning the flag off after it was on has been tested with real data in staging
-
On-call and support know the flag exists and who can turn it off
Stage 1: internal
-
{{flag_name}}is on for internal users or a test organisation only -
Team members have used
{{feature}}and reported problems are fixed or accepted
Stage 2: limited rollout
-
{{flag_name}}is on for a small share of users or a group of friendly customers -
Metrics for the flagged group are compared with the control group
-
Support tickets and feedback about
{{feature}}are reviewed -
Decision to widen, hold or turn off is recorded
Stage 3: full rollout
-
{{flag_name}}is on for all users -
Metrics stay within the agreed limits after a full business day or a peak period
-
Release notes or customer announcement for
{{feature}}are published optional
Clean up
-
Removal date for
{{flag_name}}is set and a ticket exists -
Code paths for the old behaviour and the flag checks are removed
-
Flag is archived or deleted in the flag tool after the code removal is deployed
Checklist as Markdown
Start this checklist from your pipeline
When a pull request that adds a new flag is merged, POST /api/v1/runs with template "Roll out flag {{flag_name}}" and the flag key.
curl -X POST https://runbook.metastic.ch/api/v1/runs \
-H "Authorization: Bearer $RUNBOOK_TOKEN" \
-H "Content-Type: application/json" \
-d '{"template": "Roll out flag {{flag_name}}", "variables": {"flag_name": "…", "feature": "…"}}'Frequently asked questions
What is the difference between a canary release and a feature flag rollout?
A canary release sends a share of traffic to a new version of the whole service, usually at the infrastructure level. A feature flag rollout runs the same version for everyone but turns one feature on for chosen users. Many teams use both: a canary for the deploy, flags for the features inside it.
How long should each rollout stage last?
Long enough to see the metrics you agreed on with real usage. For a busy product that can be a day. For features used weekly, it may need a full week. Decide the minimum duration per stage before you start, so the rollout is not rushed.
When should a feature flag be removed?
Soon after the feature is fully rolled out and stable. Release flags are meant to be temporary, so give each one a removal date and a ticket. Long-lived flags for permissions or plans are different and should be managed as configuration.
Updated