DORA metrics in practice, and the three ways teams break them
Four numbers that describe how a change gets from someone’s head into production, and what happens to each of them when you point it at a person instead of a pipeline.
The DORA metrics are the most useful thing to come out of software delivery research in twenty years, and also the most reliably misapplied. The misapplication is nearly always the same move: taking a measurement of a system and using it to evaluate a person. That move breaks all four, in four different ways.
Worth being precise about what they are first, because the four are usually recited without much attention to what each one is asking.
The four, and the question each one asks
Deployment frequency asks: how small is your unit of change? A team deploying many times a day is not working harder than one deploying fortnightly; it has arranged things so that a change can be released without a ceremony. The number is a proxy for batch size, and batch size is a proxy for how much risk is bundled into each release.
Lead time for changes asks: how long does a change sit still? Measured from commit to running in production, most of it is almost never coding. It is waiting: for a review, for a queue, for a release window, for someone back from holiday. This is the metric that most reliably points at something you can fix, because queues are organizational rather than technical.
Change failure rate asks: what proportion of releases cause a problem? It is the counterweight that stops the first two becoming a speed contest. Without it, “ship faster” has no opposing force.
Time to restore service asks: when something breaks, how long until it stops being broken? The research is consistent on this point and it is the least intuitive of the four for people outside engineering: teams that recover quickly outperform teams that fail rarely. Optimising for never breaking produces caution, and caution is expensive in a way that does not appear on any dashboard.
Read together, the four describe a system that can move quickly and absorb being wrong. Read individually, each one has an obvious way to cheat it, which is why they are only ever quoted as a set.
Break one: treating them as individual performance
Deployment frequency per engineer is not a DORA metric. It is a different number that happens to share a name, and it measures something closer to “who happens to work on the service with the least release friction”.
The damage is quick. Deploys get split to raise the count. Risky changes get attached to somebody else’s release. The engineer who spends a month on a migration that will make everyone else’s deploys safer posts a terrible month. And the person who notices all of this happening learns that saying so is not rewarded either.
The unit is a delivery system: a team, a service, a value stream. Attributing a deploy to a person is fine for a feed, for acknowledgement, or for a card. It is not fine as a denominator.
Break two: benchmarking against other companies
The published performance bands are useful for orientation and terrible as goals. They aggregate across wildly different contexts, regulated and unregulated, greenfield and thirty-year-old, teams shipping a web app and teams shipping firmware to devices in orbit.
The comparison that carries information is your own team against itself last quarter. If lead time has gone from nine days to four, something real improved and you can go and find out what it was. If you are at four days and a report says the top band is under one, you have learned approximately nothing about your own constraints.
Break three: measuring only what is easy
Deployment frequency and lead time come almost free from a version control system and a CI pipeline. Change failure rate and time to restore need somebody to define what counts as a failure and to record when it started and stopped. So a great many teams instrument the first two, skip the second two, and end up with a dashboard that only rewards speed.
This is the most consequential of the three breaks, because it looks like progress. Half of DORA is not a smaller version of DORA. It is a pure velocity target with the safety metrics removed, which is close to the opposite of what the research argues for.
Instrumenting them from what you already have
You need less than most tooling implies. Every one of the four is derivable from events that already exist in systems you already run.
- Deployment frequency: one event per successful production deploy, carrying the service and the timestamp. A deployment webhook already emits this.
- Lead time: the commit timestamps on a change, and the deploy that shipped it. The join is the fiddly part: you need to know which deploy carried which merge, which is why it is worth attributing a deploy to the change it released rather than to the machine account that triggered it.
- Change failure rate: an event when an incident is opened, with the release it is attributed to. This requires a decision about what counts, and that decision is the actual work. Write it down; a definition that lives in somebody’s head will drift.
- Time to restore: the open and close timestamps on that same incident. Two fields.
None of that needs a specialised platform. It needs somewhere to put events, an aggregate per team per period, and enough discipline to keep the definitions stable. If you can post JSON to an endpoint, you can instrument all four, including from an incident tool that has never heard of DORA, because a signed webhook accepts whatever you choose to call your events, and a board can rank on a metric you invented this morning.
What the four keys will not tell you
They describe the pipe, not what goes through it. A team can be elite on all four while shipping features nobody uses, and the metrics will look magnificent the entire time. They say nothing about whether the architecture is accumulating debt, whether the on-call rota is burning people out, or whether the roadmap is any good.
This is not a criticism. A thermometer is not a poor diagnostic tool for failing to be an X-ray. But it is why the four keys belong next to qualitative signals, retrospectives, on-call load, what people actually say in one-to-ones, rather than replacing them.
A reasonable starting point
- Instrument deployment frequency and lead time first, per service, because they are nearly free.
- Write down what counts as a change failure before you measure one. Circulate it. Argue about it once, then leave it alone.
- Report all four together, per team, per month. Never per person.
- Compare each team to its own previous quarter, and treat any external benchmark as orientation rather than a target.
Then leave it alone long enough to mean something. The most common failure after instrumentation is not a wrong metric. It is changing the definition every six weeks until no comparison is possible.
Related: the full guide to tracking the four keys, how to measure a team without the result becoming surveillance, and what to put in front of an executive who does not read pull requests.