When you need a high-level indicator of progress, it is important to focus on outcomes, and not to have too few metrics. No metric is perfect: they all distort behaviour as people optimise to the indicator, not the outcome. Portfolios of multiple metrics smooth out this effect. Teal Unicorn prefers a Balanced Scorecard: multiple portfolios (there are usually four quadrants) of metrics.
e.g. Here is our preferred balanced scorecard of improvement progress:
|
Value throughput
|
Sustainability
|
|
•User-story effort “points”
•Percentage of work which is creating new features
•Amount of money spent/period
•Benefits realisation
|
Culture debt
•Staff satisfaction
•Churn
•Health
|
System debt
•Improvements
•Problems
•Risk
|
|
Quality
|
Improvement
|
|
•Customer Satisfaction
•Defects
•Ease of obtaining funds
•Reliability of services
•Fitness for purpose
|
•Hours spent on improvement
•Improvement results
•Innovation
•Learning
|
|
Narrative
|
|
All reporting should come with a narrative of what we are seeing and why, prepared by those close to the work. Don’t leave it to recipients to spot and infer what’s happening.
|
The metrics above are only examples of suggested measures. Start with what you can measure now. We believe bad metrics inspire better metrics, so long as we are clear about the quality of the metrics.
One would want to have a portfolio of metrics under each quadrant of the scorecard. Be constantly improving and changing them – dashboards are a dynamic activity, not a static artefact. We also create a spreadsheet of measured, needed, desired, and rejected metrics, to direct development of the scorecard over time.
Remember, no artefact like a scorecard has any value unless you also build a practice of people and processes around it to keep it alive. Find some smart data kids: they work wonders with big data these days.
Value and Quality are lagging indicators of how well we have done at improving work, how far we have come, our “wake”. Sustainability (systems and culture debt) and Improvement are leading indicators of how well equipped we are to improve in the future.
Balanced Scorecard is like an advanced game of Whack-a-Mole. It is about keeping all four quadrants advancing. If you push Value too hard you may degrade Quality. If you improve Value and Quality too hard you may reduce Sustainability by neglecting pay-down of system debt or grinding your people, and you may forget to Improve.
From our book The agile Manager.
Here is an example of presentation layout.
