Because agility at scale is one of the crucial issues for large organizations, we interviewed four experts in order to put together an ebook, allowing to understand the benefits of agility at scale and SAFe. The following article is an extract of this ebook and allows us to discover how to measure the effectiveness of agility at scale.
To measure the effects of a move toward scaled agile practices, the usual KPIs are inadequate
To get straight to the point, identifying the much-discussed ROI (return on investment) that every leader hopes for after an agile transformation can be a tricky task. We might even say that, if agile transformation is guided only by a quest for this ROI, it often fails, as the focus is no longer on the development of the corporate culture required. To measure the effects of a move toward scaled agile practices, the usual KPIs are inadequate. We can start by looking at the impact on people, on employees’ commitment, and motivation levels. Regular interaction with teams allows us to “measure” or at least “sense” their state of mind. We can look at their degree of involvement, of sensitivity to the approach in progress, their acceptance of the right to make mistakes, and discuss these.
It’s therefore possible to measure process improvement, but drawing conclusions in monetary terms on the savings made would be a matter of guesswork. On the other hand, more efficiently managed projects, earlier deliveries, and incidents identified prior to production are more factual elements that bring real savings to the organization and help to give it a high-quality image.
If we really want to analyze this, we can assess the number of people collaborating in an agile way, the level of maturity of the teams and the prospects for growth. Remember, however, that the customer is the focus of every aspect of the agile approach, and so measuring customer satisfaction seems the best measure.
We only measure what we want to improve
The first thing to do is to go back to basics as regards the KPIs, which are there for us to learn from, as we only measure what we want to improve. Once this improvement has been achieved, you change the metric. It’s also very important that these steps are taken by those who wish to learn more. A systemic view of these measures is required. A Scrum team measures its system, and an executive measures the organization and the relationships between the parts of his or her system.
Nowadays, there are all sorts of tools that allow an organization to better understand its adoption of agile, how a product is progressing, the returns from end customers, and so on. Each metric provides a benefit for a given system. As Enalean points out, measuring revenue involves hypothesizing about a choice that will have to be validated after a production increment has been implemented.
More and more organizations are adopting OKRs—Object Key Results—as a means of measurement. In this case, the organization sets a strategic goal. We then measure the Key Results that support it. Everyone in the enterprise pursues these goals in their own way.
WE MEASURE the value a scaled organization can deliver to its customers, and how quickly
I fully agree; I’d also add one point from my own experience: the need to consider what value a scaled organization can deliver to its customers, and how quickly.
This value will be measured directly by the use of the apps delivered, and indirectly by the effect on the revenues derived from the activity supported.
For example, let’s say my team had been commissioned to implement an application for managing contracts awarded by the government. Thanks to the simplicity of the tool, its flexibility, its functional coverage, the speed of our response (rarely more than two months for the first delivery), our involvement as customers of the public sector procurement organizations, and the quality of our solutions, in the space of three years, the government entrusted us with all of its new contracts. And the business more than doubled its sales within five years.
We can define relevant indicators for each of the defined objectives
Once again, starting from the goals that have been set: why do we want to take a scaled agile approach? What problems are we aiming to solve? What do we want to improve?
From there, we can define relevant indicators for each of the objectives and measure and adjust them regularly if the objectives change, if we think we’ve achieved them, or if new ones appear.
As far as the “how” is concerned, we should base the approach on appropriate methods and tools (OKRs, measuring the quality of the code, tools that support agile practices, user surveys, mood of the teams, and so on).