How to build a leaderboard that does not create the behaviour you were avoiding

Goodhart’s law is usually quoted as a reason not to build leaderboards at all. That is too strong. A measure becomes a bad target under specific conditions, and most of those conditions are choices you make while configuring it. Here are the ones that matter, roughly in order of how much damage they do when you get them wrong.

1. Count more than one thing, and weight them

A single-metric board is a single-metric instruction. Rank on merged pull requests and you will get more, smaller pull requests, because that is the cheapest way to move the number and everybody works it out within a fortnight.

Two or three metrics with different weights are much harder to game cheaply, because the cheapest way to move one usually damages another. Weighting matters more than the count of metrics: a sales board that scores revenue at one and deal count at zero reports both, and ranks on the one that means something. Two closed deals worth $2,000 should not outrank one worth $400,000, but the deal count still belongs on the row.

2. Choose a window that forgets

An all-time board is decided in its first month, and after that it is a tenure ranking wearing a performance costume. Whoever joined first wins, permanently, and everybody else can see the arithmetic.

A rolling window (thirty days is usually right) has the property that somebody who has a good month appears, and somebody who has a quiet month is not punished for a year. It also means the board is worth looking at again, which an ordering that cannot change is not.

Keep one all-time board if you like, and understand what it is: a record of accumulated contribution, useful and mostly static. Do not make it the default view.

3. Rank teams by default, people by decision

Team-level ranking answers almost every question an organization actually has, and carries a fraction of the social cost. A team is also a much harder thing to game, because gaming it requires several people to agree to do something slightly dishonest, which is a high bar in practice.

Individual boards have a place: people genuinely like seeing their own contribution reflected, but the decision to publish one should be explicit, reversible, and recorded. If it is the default, nobody chose it.

4. Enforce visibility where the data is

If who appears on a board is decided by the interface, then the policy is decorative. The first API call, the first CSV export, the first debugging session with the network tab open, and the rule is gone.

Put it in the query. It is a small amount of work at exactly one place, and it converts a promise into a property. It also means an organization that switches to a restrictive mode does not have to trust that every future page was written carefully.

5. Make the ordering total

A detail that reads as pedantry and is not. If two people have the same score and the comparator has nothing left to say, the database is free to return them in a different order each time. Users read that as ranks randomly swapping, and it destroys confidence in the whole board faster than a wrong number would, because a wrong number looks like a bug and a shuffling rank looks like the system is arbitrary.

Chain the tie-breaks and end with something stable, an identifier, so the order is deterministic even when the scores are identical.

6. Give invisible work a route onto the board

Every leaderboard built purely on version-control events tells the support engineer, the technical writer, the person who runs incident review, and half of the design team that this is not for them. Which is fine if you meant it, and corrosive if you did not.

The fix is not to invent proxy metrics for invisible work: that goes badly, and usually ends up measuring who is best at logging their own contributions. The better move is to let teams define what counts in their own terms and send it. A board that can rank on any metric you can emit is a board a support team can use, because “resolved_escalation” is as valid a metric as merged. Metric keys reach the query as parameters, which is the boring implementation reason a customer can invent one without waiting for us to ship it.

7. Decide what happens to the person at the bottom

This is the question that most implementations never ask, and the one that determines whether the board survives its first difficult conversation.

Somebody is last. If being last has consequences, you have built a performance-management instrument and should say so plainly, because people will find out and the discovery is worse than the disclosure. If being last has no consequences, make sure that is visibly true. This usually means limiting the board’s depth so it shows a top group rather than a full ordering, and never referencing position in a review.

A depth limit is a cheap and effective intervention. A board showing the top twenty-five is a celebration. The same board showing all four hundred people is a ranking of an entire organization, and they are not the same artefact even though the query is identical.

The short version

• Several weighted metrics, never one.
• A rolling window that forgets, not an all-time total by default.
• Teams by default; individuals by an explicit, logged decision.
• Visibility enforced in the query, not the interface.
• A total ordering, so ranks never shuffle.
• Any team can define a metric and rank on it.
• A depth limit, so the board is a top group rather than a full ordering of everybody.

None of this makes a leaderboard immune to Goodhart. It moves it from “will certainly produce bad behaviour” to “produces bad behaviour if you stop paying attention”, which is the most any measurement system achieves.

Related: which metrics survive becoming targets, why most workplace gamification fails, and how to keep measurement from reading as surveillance.