It’s an exciting time: you’ve gained buy-in for your agile transformation and have secured support from management. You’re now involved in a major change initiative, maybe even seeing the first signs that it’s producing results. But how do you measure your progress to know that you’re achieving the coveted “agile at scale?”
In our consulting work with clients such as Apple, Mozilla, and Roche, we’ve seen too many companies measure the wrong things. Often, they tend to measure the results of a long process rather than the levers – or predictive metrics – that drive those results.
Consider a football team. Their goal is to win games by scoring more points than the other team by getting touchdowns and field goals. But a football team that measures only its wins and its points scored won’t get very far. Football coaches track all sorts of obscure behaviors that predict team success, like the quarterback’s release time, the time between when the quarterback receives the ball and when he throws it. Factors like release time are levers that drive results. A fast QB release time correlates with completed passes that move the ball, leading to scoring more points, and wins on the field.
So the goal is to track not only wins themselves – i.e., the result you wish to achieve – but also the behaviors that tend to predict those wins. Smart companies (and smart coaches) measure the levers that drive the results they wish to achieve.
We call these levers predictive metrics – leading indicators that target a specific behavior that, in turn, drives an outcome you wish to produce. With predictive metrics, you measure what’s happening now that will drive your implementation in the future.
Why measuring agile implementations is hard
Agile implementations might involve a two- to five-year timeframe. Of course, you’ll measure incremental results along the way; but you also need clear, early indicators that you’re on the right track. A year or more is too long to wait to see the results of your efforts. The alternative is to look at a few key levers that demonstrate that your organization’s behavior has started to change.
For most organizations, agile transformations involve a large, systemic change. It takes time. Mature companies may not have seen change of this magnitude in years, and many key players weren’t anywhere near the company the last time such a radical shift occurred. Less mature companies may never have seen the scale of change involved in an agile implementation.
The change you’re initiating to scale agile is exponentially more difficult and uncertain than applying the agile toolkit to a dev team. While agile software development has become common practice, scaling agile across the entire enterprise is a more recent development, and there are fewer examples of companies doing it well (and therefore less tribal knowledge). A radical change to the way an enterprise operates requires a large transformational shift in culture, new ways of working, new tools, and top-down leadership. The way forward is uncertain, making it hard to define even the targets you’re aiming for, let alone the best measures to capture your progress.
At the same time, approaches that measure the results of a process are often onerous, taking far too long to track, while not providing the indicators that allows managers to make incremental course corrections that will support the implementation. Too often in our consulting work with clients, we’ve seen multiple people measuring the same things in different ways, resulting in incoherent readouts for managers.
Predictive metrics: measure behaviors, not results
Change occurs when people behave differently. By measuring behaviors, you can track a small number – three to five – of key indicators that demonstrate change is happening within your organization. When organizational behavior shifts, the change you envision is already taking place. Predictive metrics are fundamentally different from results metrics in that they measure a process or behavior rather than a result. By measuring the micro behaviors that, repeated multiple times, result in macro changes, you ensure that your metrics matter.
FYI
Predictive metrics provide a direct line of sight between your implementation plan and its execution. They enable you, as a change leader, to seed improvements throughout the organization.
Measuring three to five key behaviors also puts your focus as a change agent on the most critical areas, enabling you to make the incremental adjustments that lead to better, faster decisions. Further, predictive metrics establish an early warning system that helps you get drifting projects back on track.
There are three components of a predictive metrics system:
- Identifying behavioral levers
- Designating a simple, frictionless metric for these behaviors
- Creating a target for this metric
Very few agile transformations show immediate success, so you need a target goal that focuses your attention each and every week, or else you’ll have to rely on a “hail Mary” pass six months from now.
Levers that drive behaviors
So how do you identify the levers that drive behaviors? Consider the result you want to achieve – the future state you envision – and reason backwards.
Consider this simple example from a large tech startup. The company wanted concrete measures to drive its valuation in its D Round funding. Below is a diagram that shows their objectives related to funding and the key results per objective the company strives to achieve. In green are the predictive metrics – the leading indicators that drive each objective.
One of this company’s goals was to have 10 new hires by October 10th. To predict if they’ll get there, the company might measure the number of hires per week, but a more predictive metric is the number of resumes screened because this metric measures the behavior that drives the objective. Measuring the number of new hires tells you when you’ve gotten there; measuring the number of resumes screened predicts that you will get there.
Having identified a key lever that drives behavior, the company then needed to estimate a desired rate of increase for it. In this case, the predictive measure is the number of resumes screened per week. It is the behavior of screening a certain number of resumes per week that predicts the desired result. Measuring the number of hires tracks the key result of your activity, but measuring the processes involved in hiring, such as screening resumes, is a better indicator that you will hit your target. Measuring what you’re doing today (screening resumes) predicts the result you will produce tomorrow (10 new hires by October 10).
In another example, a very well-known large tech company used predictive metrics to drive a major change initiative. After mapping out a new process, piloting and phasing it in with a subset of teams, they looked at the levers that would drive the desired behavior: compliance with a new release planning process. What they found sounds trivial, but it worked: they measured the number of teams that had assigned the release plan to a sprint. They reasoned that the first evidence of release planning was to get it in the backlog and associated with a sprint.
The teams that slotted the release plan into their backlog and earmarked a sprint number devoted to the next release plan walked the talk – they were adopting the new planning process. The goal was to have 95% of the teams in this large organization compliant with the new process, and the company found that simply scheduling the next release plan predicted the adoption of the wider process, much more than simply attending a training session. This metric captured intent.
Earliest possible indicator
Predictive metrics give managers the earliest possible indicators that a change initiative is on (or off) track. They provide a direct line of sight to the goals of your agile implementation, enabling you to act swiftly if your change initiative’s objectives begin to drift.
Keys to success with predictive metrics:
- Define the key levers that drive behaviors and make them quantifiable.
- Measure them early and often: think in days and weeks, not months and years
- Make your predictive metrics simple and easy to deploy: no IT implementations to get them done, no lengthy meetings and post-mortems
- Above all, these metrics need to demonstrate that the change you desire is occurring.
The adage about being careful what you wish for goes double for metrics.
You’ll get what you measure.
Track and measure the processes and behaviors that drive broader results. This small set of measures then becomes a day-to-day dashboard that tracks and demonstrates your success and makes the case for your agile transformation throughout the wider organization. Your scaled agile implementation is too important to leave to time and chance. With predictive metrics, you’ll know right away when your organization is headed for success.