Agile Metrics

Agile KPIs and ROI

Before we jump into exploring Agile metrics to measure teams specifically, it’s important to remind ourselves that Agile projects are still beholden to good old fashioned measurements of success too.

Business metrics, KPIs (key project indicators), and basic proof of return on investment (ROI) still matter in an Agile environment, so we have to keep an eye on those metrics as well as our Agile metrics. These data points are particularly important during an Agile pilot, when you’re looking to prove the impact of a serious change, namely the switch to managing work using the Agile methodology.

If you haven’t previously been tracking basic performance metrics for your team, it’s going to be tough to quantify the effects of an Agile implementation. So even before you choose the right Agile metrics, make sure you’ve laid the foundation for measuring your marketing.

Five Agile metrics worth measuring

Now that we’ve covered the basics, it’s time to get into metrics for Agile teams. We’re going to cover five different options, most of which will apply to just about any kind of team. For those that are best suited to one particular methodology, we’ll be sure to note that during the explanation.

When choosing which of these Agile metrics to track, remember that it’s difficult to optimize something you don’t measure. But you also don’t want to waste hours of time on pulling numbers. Consider what you’ll do with the data you collect, and whether your current tool set provides you easy access to the information you’re after. If you don’t plan to act on your metric, or it’s complicated to collect, you may want to try another.

Agile velocity

Agile velocity simply measures the amount of work that a team completes during a set amount of time, usually a sprint or iteration. It’s usually gauged in points, which means it requires the team to size each piece of work that they put into their sprint backlog.

Any Agile team that uses timeboxed iterations can measure their velocity. This means both Scrum and Scrumban teams can easily make use of this Agile metric, while it doesn’t often work very well in a Kanban implementation.

At its core, the goal of measuring velocity is to see whether the team is improving over time, but it can be misused. If velocity becomes a bludgeon with which to beat the team, they’re likely to start gaming their estimations so they appear to be doing more even if they aren’t. Make sure velocity is simply a health check for the team, not an unreasonable expectation.

As you track velocity, look for any erratic moments. If these become the norm rather than the exception, help the team investigate the root cause. Some possible retrospective questions might include:

• Are there unforeseen challenges we didn't account for when estimating this work? 

• How can we better break down work to uncover some of these challenges?

• Is there outside business pressure pushing the team beyond its limits? 

• Is adherence to best practices suffering as a result?

• As a team, are we overzealous in forecasting for the sprint?

Burndown chart

Velocity is a trailing indicator—it won’t give you any data about an iteration until it’s already over. A burndown chart, on the other hand, gives you day-to-day data about how the team is doing throughout the sprint.

Like velocity, an effective burndown chart requires estimation of the projects being used.

Here the team has committed to 160 points, and as time goes on and they complete some of their work, the chart “burns down” until the sprint is over. At that point they’ll hopefully have zero work remaining and the vertical bars will have disappeared.

Burndown charts are highly useful on young Agile teams who need some guidance on how much they can really get done within each sprint. They can tell us if:

• The team isn’t committing to enough work because their burndown chart “burns down” early sprint after sprint.

• The team is committing to too much because they have multiple points left on their chart at the end of each sprint.

• The team isn’t breaking work down into small enough pieces because the burndown line makes steep jumps rather than a gradual decline.

• External interruptions are derailing the team because the vertical line representing points goes up mid-sprint as new work gets added.

Once a team stabilizes and delivers a steady amount of work the burndown chart may become less useful. Of course, it’s hard to get these kinds of metrics retroactively, so keeping them on the radar can help surface problems before they get out of hand.

Whitepaper: How to Become an Agile Agency

Datasheet: The Agile Marketing Cheat Sheet

Quarterly/strategic burndown chart

Burndown charts provide insight into how things are going during a particular iteration, but we also need to make sure we’re incrementally moving towards bigger goals.

On a traditional Agile software team, this trajectory could be tracked via an epic or release burndown chart. On an Agile marketing team, they prefer to call it a strategic or quarterly burndown chart. This is a longer term look at what the team is doing and how it’s related to the strategy over time.

Image source: Atlassian.com.

You can see in the chart above that a few new items have been added to the team’s larger backlog after each sprint. While we wouldn’t allow that to happen to a sprint backlog in the middle of an iteration, it’s completely acceptable in this context.

Over time managers and leaders may decide to add or remove some items based on what they learn, or based on changing organizational objectives. As that happens the strategic burndown keeps everyone aware of “the ebb and flow of work” within the long term strategy.

Monitoring the strategic burndown chart can give you insight into whether the team’s work is actively contributing to larger goals, or if they’re just churning through busy work.

Red flags to watch for on this chart include:

• Strategic forecasts aren't updated as the team delivers work

• No change is visible after several iterations

• Chronic scope creep, which may be a sign that leaders don’t fully understand the problem the Agile team is trying to solve

• Strategic scope grows faster than the team can absorb it, meaning they’ll never be able to complete a strategic goal

Cycle time

So far our Agile metrics have been mostly concerned with work at the team level, but we may also want to know more about how long it takes the team to finish smaller projects. Cycle time can deliver this insight

Cycle time simply measures how long it takes something to get done from start to finish. Joel Bancroft-Connors of AgileConnection calls it “hands on keyboard time.”

Measuring cycle time is best done automatically via your Agile software or tool of choice, but even measuring manually alongside a physical task board or Kanban board will give you useful data.

Just keep track of how long a project/task/story takes to get into your “done” column once it’s pulled from the backlog. The average for all your work is your cycle time.

It’s measured at the level of a user story (or a task or project, depending on how you structure your Agile board), so it’s a much more granular metric than anything we’d see on a burndown chart.

Cycle time also gets us much more immediate feedback, because we can see the results of any changes right away. When we adjust the system, cycle time will either increase or decrease right away, so we know if the experiment succeeded or failed in a short time. Our goal is cycle time that’s both consistent and short, regardless of the kind of work being done.

Consistent cycle time means we can accurately predict when we’ll be able to deliver individual pieces of work, whether we’re using continuous flow or sprint-like timeboxes to handle larger projects.

Agile team happiness

Our final Agile metric isn’t strictly related to the Agile process at all. Instead, it’s a look at the team’s happiness.

There’s no need for any fancy tracking system here. Just start your retrospectives by asking each member of the Agile team to score their happiness on whatever scale you choose. Then keep track of the average from one retrospective to the next.

Happy teams create better work, which will deliver more satisfied customers, so this is in fact an important business metric. But it’s also important to recognize that if all your Agile process metrics are perfect but the team is deeply unhappy, you’re headed for some trouble. The team may be masking its issues while maintaining high performance, meaning burnout and high turnover could be on the horizon.

New team members invariably disrupt an Agile team, so we want to keep people happy and the team configuration stable whenever possible.

Whitepaper: Agile Marketing for Creative Teams

Webinar: What's Keeping Marketers From Going Agile?

How long to track Agile metrics

You may decide to track one or all of the metrics we’ve discussed so far, but whatever combination you select make sure you get more than a brief snapshot. Track metrics for at least six months to identify larger trends.

That doesn’t mean you can’t take any action on your findings until six months have passed (that wouldn’t be very Agile). But this ongoing look at different data points can help you see how one impacts the others.

Agile metric pitfalls to avoid

Before you start measuring Agile metrics, consider doing the following:

• Keep Agile metrics within teams. Neither teams nor managers should compare metrics across teams, and that goes for everything from velocity to happiness. These metrics are for individual teams only.

• Measure, don’t target. When a measure becomes a target, it ceases to be a good measure. The goal of Agile metrics is to spot and understand trends, not to create magic numbers the team tries to hit at all costs.