Tel: 001-703-945-6060

Email: rk@myagilegurus.com

  • LinkedIn
  • Facebook

United States

United Kingdom

Canada

India (coming soon)

© 2020 by Agile Gurus LLC

Search

Metrics: The good, the bad and the ugly

Updated: Oct 12, 2019

As organizations transition into the agile space, change agents, leaders, teams and senior management all look towards metrics as a way to measure progress and adaptation of the new methodologies.

Whatever your feelings on metrics, organizations will expect them for your team. You don't want to measure only one aspect to the detriment of other information, but you also don't want to measure too many things and scatter your team's focus.

Before you start going down the slippery slope of metric creation, consider who the audience for your metrics are. For e.g., though velocity is a great metric for teams to reflect upon during their retrospectives, with an eye towards getting better, it is an awful metric to report onto senior management. It introduces quite a few anti-patterns which could eventually lead to teams in a portfolio or program being compared to each other.

Another important rule of thumb would be to stay away from tracking a single metric for any given team. If you only measure one key metric, it is easy to get tunnel vision. Be it the teams focusing on just making the metric better (often through gaming the system) or management using the measure to drive all decisions, you can end up with a product or organization that looks good but is really driving off a cliff.

So what are some great metrics to track? Here are my four go-to metrics for an agile team, along with some experiences on their effectiveness


Cycle Time Cycle time is your direct connection to productivity. The shorter the cycle time, the more things are getting done in a given time-box.

It is the time spent working on an issue-typically, the time taken from when work begins on an issue to when work is completed, but it also includes any other time spent working on the issue. For example, if an issue is reopened, worked on, and completed again, then the time for this extra work is added to the cycle time. In software terms, I tend to think of this as "hands on keyboard" time. Measuring cycle time is best done automatically via your agile life-cycle tool of choice (Jira, Rally, VersionOne etc) , though even measuring with a physical task board will give you useful data.


Escaped Defects This measure is the connection between customer satisfaction and the team. The lower the defect rate, the more satisfied the customer is likely to be with the product. With a high escaped defect rate, even the most awesome product is going to have a lot of unsatisfied customers. The ultimate goal of any team should be to see a reduction in the number of escaped defects over time for a given release which should show you improved code quality.

There is no need to formally track the bugs identified within a given iteration. The thinking is that all of those should be resolved before the end of the iteration or the story will have to "carry over" into the next iteration for completion. It is critical, however, to track any defects that are identified AFTER an iteration ends. These are called "escaped defects" and are a serious signal that something has gone wrong with the delivery of the team.

At the end of every iteration, teams should measure the number of defects identified in their formal defect tracking backlog and trend whether that number is increasing or decreasing. If increasing the team has some major change to make in its delivery stream (addition of code reviews, test automation etc.).


Planned-to-Done Ratio This metric is a way to measure predictability. If a team commits to thirty stories and only delivers nine, the product owner has about a 30 percent chance of getting what they want. If, on the other hand, the team commits to ten stories and delivers nine, the PO has roughly a 90 percent chance of getting what they want.

Measuring is a simple exercise of documenting how much work the team commits to doing at the start of the iteration versus how much they have completed at the end of the iteration.


Happiness This is the team "health" metric. It creates awareness that puts the other three metrics into better context. If all the other metrics are perfect and happiness is low, then the team is probably getting burned out, fast.

Build this into your sprint retrospectives. Open every retrospective with the team writing their happiness scores on whatever scale you choose. Track these numbers from sprint to sprint to see the trends.


28 views