[Cycle times] Improving throughput calculation (based on Github label change)

adobe / brackets

An open source code editor for the web, written in JavaScript, HTML and CSS.

http://brackets.io

MIT License

33.25k stars 7.63k forks source link

[Cycle times] Improving throughput calculation (based on Github label change) #8088

Closed peterflynn closed 9 years ago

peterflynn commented 10 years ago

Original Subject: Throughput calculation appears to be wrong

Copied from PR comment:

I think the cycle time data shouldn't actually factor into the throughput calculation at all. For example:

If we finish 5 cards in a week in serial, each card has a 1-day cycle time -> avgCycleTime === 1
If we finish 5 cards in a week in parallel, each card has a 5-day cycle time -> avgCycleTime === 5

The throughput should be the same in both cases: 5 cards per week. But the equation in getCategorySummary() will instead give very different answers for these two cases...

peterflynn commented 10 years ago

NJ & I came up with this suggestion for how to calculate it:

Find the earliest start date of any card completed in the sprint
Find the latest end date of any card completed in the sprint
Calculate the number of business days in this entire range
Divide total number of cards completed in the sprint by number of business days, then * 5 to convert to business weeks

redmunds commented 10 years ago

Re-assigned to me.

redmunds commented 10 years ago

@peterflynn I brought this up before, so I'm glad we're finally getting around to discussing it. The algorithm suggested here eliminates the overlap between cards within a release, but there could still be date ranges for cards that overlap across releases.

For example, a card for current release that was pushed from release to release could be 4 or 5 months old (e.g. "PR: #6409 Switch language / syntax mode of current document" card in current release) and really skew throughput for release.

Using most recent action date for the "end" for this release is reasonable. What we want for the "start" date is the "end" date of the previous release, right? This will eliminate overlap of dates across releases. We already prompt for "Ship It!" column for current release, so we can add a prompt for "Ship It?" column for previous release as well. How does that sound?

cc @njx @pthiess @dangoor

dangoor commented 10 years ago

@pthiess and I spoke about throughput a couple of days ago. My understanding of throughput is that it has nothing to do with releases whatsoever.

Throughput is "number of cards per X amount of time" where we could choose to make X a week or a month. So, to calculate throughput you just need to look at how many cards landed in the done column over a given period. We'd then average that over time.

We have the extra variable of two different card sizes, but otherwise it's the same idea.

redmunds commented 10 years ago

@dangoor

My understanding of throughput is that it has nothing to do with releases whatsoever.

Correct, but we just happen to organize our cards around releases, so it's convenient to use that info while we're calculating cycle times.

We could make a separate report where input is start and end time, and then search through all "Ship It!" columns for Cards that were complete during that time range. This would be addition work, but will be (somewhat?) more accurate.

pthiess commented 10 years ago

@redmunds

Correct, but we just happen to organize our cards around releases, so it's convenient to use that info while we're calculating cycle times.

I believe we shouldn't intermingle releases and throughput calculation. As @dangoor mentioned the two are independent. If the throughput calculation is based on releases we should fix that. The release should just be a random point in time when we review the outcome. The idea behind Kanban as I understand it is that the focus is on throughput and not on time boxing (as opposed to sprints in SCRUM). It is not so important at which date something is landing, it is more important that it is landing as soon as possible after it has been prioritized and started. This is my understanding of optimizing the Kanban flow. I'm inclined to put the release 43 milestone on this since we need accurate data.

dangoor commented 10 years ago

We just discussed this briefly. There's some work to be done to make this happen because it's different from how the script currently works. Before doing that work, we should decide as a team whether we're ready to move to Waffle or if it's worth investing more and sticking with Trello for a while.

dangoor commented 10 years ago

Leaving needs review until after we've had that conversation.

pthiess commented 10 years ago

Added this to Nominations as I need to discuss the timing with @ryanstewart. Instead of doing the fix for Trello we want to do this so that it works with github / Waffle.

pthiess commented 10 years ago

Definitions:

Throughput is the number of issues that were on the board (beyond Ready) that were closed per week
Cycle time is how much time it takes to go from moving out of Ready to being ready to ship (closed) Ship it

redmunds commented 9 years ago

@peterflynn @pthiess Since we're now using Waffle this issue is obsolete. Closing.