What metrics I use to measure engineer’s productivity
There is a saying by W. Edwards Deming — “You can’t manage what you can’t measure”. In software engineering, for any manager, its vital to setup correct metrics first so that a project can be well managed.
The standard SLM products like JIRA and Azure DevOps (alias VSTS) provide most of the metrics out of the box. Some of the metrics I find extremely useful to track engineer’s output are (assuming that most of the engineering community use agile) -
- Sprint Velocity
- Cycle Time
- Lead Time
- Number of Pull Requests [PRs] (controversial)
- Sprint Velocity: This metric allows a manager to see how the team has been progressing in terms of output from one sprint to another. Some points to remember are-
a. Sprint Velocity is directly proportional to the number of team members and number of working days in a sprint.
b. In general, having a bigger number is good but one needs to be realistic
c. A falling number indicates problems that the team could be facing to achieve its goal
d. It should NEVER be used for comparing the performance of one team with another because this metric is very specific to a given team.
In the diagram below, one of my project’s avg velocity of last 5 sprints is 54.
2. Cycle Time: Cycle Time is defined as the time spent for a particular task from Started to Completed. This metric allows me to track in general how many days it takes for a particular task to complete. Lower number is better (of course it cannot be zero though).
In the diagram below, one of my project’s Cycle Time is 11 days (over 90 days)
3. Lead Time: Lead Time is defined as the time spent for a particular task from Created to Completed. Its very similar in interpretation to Cycle Time except that its measured from Created date. Lower number is better. In reality, the Lead Time will almost always be greater than or equal to Cycle Time.
As a manager, my objective is to minimize the gap between both these times and get Lead Time as close as possible to Cycle Time.
An unreasonably high value of Lead Time in comparison to Cycle Time could indicate that the team is overloaded and hence cannot pick the task to work upon &/or it could mean that the product owner (or business analyst) has done the project’s planning well in advance and engineering did not get a chance to work upon it.
In the diagram below, one of my project’s Lead Time is 28 days (over 90 days)
Note: Its important to measure both these metrics over same time period.
4. Number of Pull Requests: This is one of the most controversial metric. Many in our community are of the opinion that PR count does not truly reflect an engineer’s productivity. And I agree with it partially. There could be a number of reasons that PR count is high — a developer raised a PR but that got reverted back, a developer raised a PR that never got merged into master and hence not to production, a developer raised a PR for trying out something and got discarded, et al. However I still like to be aware of this is because it gives me a rough idea of who’s actively contributing to the code repository. More importantly - I get to see who is NOT contributing to the code. This indicates that an engineer is blocked with many meetings/discussions which provides very little opportunity to work on code. This is typically true for Tech Lead-kind of roles.
Having a low count of PR for a non-Tech-Lead member should be a cause of worry for a manager.
In the diagram below, I plotted a leader board of top PR contributors for a project over 3 months period. I used this chart to discover the problems related to people’s behaviors (like spending too much time in meetings/discussions which remain open-ended), as indicated in red, which never got captured any where else. I used this chart to fix the issue of non-productive time spends on the project.
Congratulations for making this far. Show some love for this article if you liked it. I will be happy to hear what metrics you use to measure engineering productivity. Tell me your comments.
