The problem with velocity
Velocity is one of the most common metrics in software development, but very often is misused. Here's what the process managers get wrong and how to use velocity to the benefit of the whole organization.
What does velocity measure
One of the most common practices in software development is estimating tasks (or so called "user stories" – terrible name, won't use it here). There are many different ways to do it, but the most common way I've seen is:
- have the whole team discuss the task at hand and agree on estimation (following some subset of Planning Poker rules)
- use a certain set of numbers to represent estimation values. The most common set I've seen is the first few numbers in Fibonacci sequence: 0, 1, 2, 3, 5, 8, 13... up to some defined maximum. An alternative to numbers is using t-shirt sizes like S, M, L etc. and though I believe it's better, it's way less common, so I'll stick to numbers in this article.
As an outcome of the planning process we end up with key-value pairs, where every task has assigned a certain number called "story points" or just "points". This numbers is meant to represent... something. Let me get back to this "something" in a moment. For now let's assume it's a positive value that the team gains once the task is complete.
Following the planning, the team works in fixed-length cycles, let's say 1-month long. At the end of every month they look at the tasks that were delivered, they add up the number of the points of each task, and end up with a single number. After a few cycles, the team calculates the average of this number in the last few months. Let's say the team delivered 90 points in January, 70 in February, and 85 in March. The average of these 3 numbers is 82, and that's the "velocity" of the team.
This number represents how "fast" the team is going. It's a single number that represents all the team's work over a few months, and this number can be used to measure whether the team speeds up or slows down. It can even be used to compare performance of individual developers, to see who's driving the team's performance and who's lagging behind. Magic ✨ right?
Well, not really. Velocity can provide value, but the way it's often used by Agile aficionados and inexperienced managers ruins the value of this metric. Let me explain how process managers fuck it up and how to actually use velocity effectively.
How process managers misuse velocity
There are many pitfalls of using velocity. In this article I'll focus on the following 4 ways in which process managers misuse it:
- They have poor definition of story points, and often use points to indirectly represent time
- They believe that "more = better" and they push teams to increase velocity
- They use it as the only metric of team performance
- They use it to measure performance of individual team members
Pitfall 1: poor definitions
Let's start with the basic – what does a single point mean? What are we actually estimating when giving tasks certain value?
My time is not your time
One of the most terrifying, and yet common, ways to estimate is to use time. Time to finish the task. Developers are asked to predict how much time a task will take them. So 1 point might be a proxy for half a day, or a day. Then 2 points are 2 days, 5 points are a week.
In one of my old videos I explain why this makes no sense. You can watch it below or keep reading for a summary:
There are multiple things that are wrong with this approach. The main is that how much time I need to complete a task is different than how much time you need. So when we choose a single value, whom should we listen to? A senior developer who's built the product from scratch? A mid-level developer who's still learning the tech stack? Or a junior that joined 6 months ago? When asked to estimate we need to have the same unit of measurement, and when we're asked how much time it'll take to complete the task we ask ourselves "how much it will take me to complete it", hence every team member asks a different question.
Some teams try to workaround this by assigning the tasks before estimating, and during estimation they ask "how much time it will take this specific developer to complete the task?" This approach is so bad it deserves its own article, but just to mention its biggest sin – it shifts from team-oriented to individual-oriented work, where everyone only cares about the tasks they have assigned to their name, instead of caring about all the tasks the team committed to.
What to estimate instead?
The only way to make estimating work is to have a common definition of what we estimate and what each value means.
For example, what I usually ask my teams to estimate is a mix of complexity and uncertainty. But because we don't have a clear definition and a way to measure complexity, we need to come up with something on our own. This is by design. Story points are abstract, imprecise and inaccurate units of measurement. We know that they don't refer to anything tangible, we know we can't use them to measure down to the tinies detail, and we know they're error-prone. That's alright, as long as we are trying to estimate the same thing, over time we'll make our estimations better.
Hence in order to start estimating tasks, we need to meet 2 conditions:
- everyone needs to consider the same thing – if we ask a group of people "how much time it'll take you to drink this bottle of water?" everyone will think about their own capability, but if we ask "how big is this bottle of water?" everyone will think about the same thing
- everyone needs to understand the unit of measurement – since we don't have equivalent of liters or kilometers in measuring task complexity, we need to give people some guardrails so that they know how to use the units they're given (we can use techniques like reference tasks or comparison)
Pitfall 2: more = better
The 2nd way in which process managers screw up the concept of velocity is that they believe that the velocity should grow over time. I don't really blame people here, in the end velocity = speed + direction, and the faster we move the better, no? Wrong! Expecting developers to increase velocity over time contradicts the whole idea of estimating the complexity of tasks.
The base of this wrong assumption is that as the team gets better, they will deliver more value over time. And that's true, the team will build similar things faster, hence they will achieve more in the same span of time. But at the same time, as the team gets better and builds more sophisticated software, they will give lower estimations to similar tasks.
Let me give you an example. We're building an e-commerce solution that up until now supported just 1 payment provider, let's say Stripe. Now our customers want an alternative, so we add an option to pay with PayPal. Currently everything is hardcoded to use Stripe flow, hence adding new payment provider requires a lot of changes. We estimate this task to be worth 13 points. Fast forward a few months, a similar request comes – this time we need to add support to Alipay. But since we had to switch from 1 to 2 payment methods last time, we replaced hardcoded values with nice interfaces and polymorphism. Hence we estimate this task to be worth only 5 points. A very similar task has become simpler for us, therefore we assign it less value.
This is why the velocity of the team over time should reach plateau and stay roughly the same, not grow (unless the team gets more people). If the process manager sees that the velocity is consistently growing despite no changes in the team, this should raise suspicion.
Pitfall 3: using velocity as the only metric
The third mistake is relying purely on velocity when measuring team's performance. As the famous Goodhart's law kind of says:
When a measure becomes a target, it ceases to be a good measure
Hence, when a manager tells developers they need to improve their velocity, they don't incentivize the team to work harder or smarter, they just incentivize the team to... increase velocity. What that means is that the team will start inflating numbers, giving larger and larger values to the same type of tasks over time, just to show that the line on the velocity chart always goes up.
There's more, though. If the only measurement of success is velocity, then nothing else matters. Quality? Who cares, approve that Pull Request and let's go! Bug fixes? Only if a bug is assigned a number of points, otherwise it goes to the bottom of the backlog. The team works long hours and can't keep it going for long? Doesn't matter, the velocity must go up!
Pitfall 4: measuring individual velocity
The last problem I see is that managers use velocity to measure individual performance. This not only creates wrong incentives, but also might introduce conflicts within the team.
Once the team members start being evaluated by the number of points they deliver, they lose incentive to work as a team. Any activity that does not lead them to getting more points loses importance. Pair programming? Nope, because the task is assigned to another developer. Code review? Sure, as long as the other team member returns the favor. Oh, and don't expect a solid code review, just a quick "LGTM". The same with any other activity – if my performance, and therefore salary, bonus, and maybe even promotion, is based on the number of points I deliver as an individual, then I'm going to optimize for this.
What's more, this creates a competition between team members. You see, in Agile estimation, 2+2 is not necessarily equal to 4. A huge 8-point task comes with a lot of uncertainty and can take the whole week, while a 2-pointer might take just half a day. In order to maximize individual performance, everyone will try to pick the simplest tasks, leaving the more challenging ones to anyone who fails to play the performance game.
Imagine if the most experienced developer in the team, the one that helps everyone around, the one that answers tens of questions everyday, the one that maintains the high bar for code quality and who drives the technical decisions, suddenly stops and his answer to every question is "sorry, have to focus on my task". That's what using velocity to measure individual performance does to the team. And when that happens, everyone loses – the developer, the whole team, and even the process manager that came up with this idiotic idea in the first place.
The actual value of velocity
Alright, that's enough of ways to screw up the teams. Ok, just one more – never compare velocity across multiple teams. Every team has their own definition of points, hence every team has its own velocity.
Ok, now I'm done, for real. So let me explain how to actually use velocity in a way that works for everyone – the developers, the process manager, and the stakeholders.
Predictability, not productivity
First, stop trying to maximize velocity. Instead, try to minimize its volatility. If your velocity is 40 points, but you delivered 100 points 3 months ago and 10 points for 2 months afterwards, your team is unpredictable.
What it means is that process managers don't actually know how much their teams can deliver, so their promises to stakeholders can't be trusted. A team that delivers 40 points month after month is a better partner for the business than a team that sometimes delivers 100 points and sometimes 10. Business wants predictability and this is where velocity can help, if used the right way.
Combine with other metrics
Since velocity only describes the "quantity" of work done, it does not offer enough for team managers to understand how their team is doing. Maintaining the current velocity might be coming at high cost: maybe the team sacrifices quality? Maybe they ignore testing? Maybe they need to work long hours?
In order to understand the team better, managers need to look at a number of metrics and find a healthy balance between them. The exact set of metrics will vary from team to team, but at least we need to observe 3 qualities: the quantity of work, the quality, and the sustainability (or team happiness). If we ignore either of these 3, eventually we will get our team in trouble.
Make the team work together
Lastly, bring the team together. Estimating tasks by individual team members, measuring individual velocity, assigning tasks in advance – all these practices incentivize team members to focus solely on their own work. In order to build a high performing team, process managers need to change this incentive and focus people on working together. Whether it's pair programming or code reviews, or just asking each other a few questions, developers should be comfortable helping their teammates without worrying about their own velocity going down.
Listen to your team
As the last advice in this article, I strongly recommend every process manager to listen to their team. If you manage a team, ask questions about the process during 1-1s with your team members, ask them during retrospective, ask them casually over lunch. Most of the pitfalls I mentioned above are obvious to team members and they will tell you what confuses them and what doesn't work. You don't need to be an Agile expert (it's probably even better if you're not) to create a high performing team. You need to listen to your team and update your process so that it allows them and encourages them to do their best.
And here's an infographic with 4 pitfalls of velocity that you should avoid: