This article discusses 2 modern frameworks of setting up software development processes: Just-in-time and Theory of Constraints. First, we'll discuss how to manage tasks and hand them over between different steps (Business Analysis, Development, Testing) and how/when to release changes to production. Then we'll touch on their relationship with the Continuous Delivery: we'll talk about branching strategies, release readiness and quality considerations.
Pull vs Push systems
Just-in-time (JiT) and Theory of Constraints (ToC) are often mentioned together because both are Pull systems. Meaning that if you have multiple steps in your process (e.g. in software projects we have Dev followed by Testing), the next step (Testing) tells the previous step (Dev) when they’re ready to take on new tasks. Only then Devs can publish their work (e.g. push into the trunk).
This is different from the Push systems - when Devs can publish whenever they finish their job. Push systems are awkward. If Devs submitted 5 tasks into testing, then we can’t deliver the new changes until all 5 tasks are tested. This leads to:
Slower deliveries. The 1st task has been tested, but can’t be delivered because the other 4 tasks haven’t been tested.
Feature freezes. If Devs push yet another task, now it’s 6 tasks that block the delivery. This requires a dedicated period when no one pushes anything.
Larger risks because of the coarse-grained deliverables. The more changes we deliver, the higher the chances something very bad happening.
ToC vs JiT in theory
Okay, so both Just-in-time and Theory of Constraints are Pull systems, what’s the difference then? Well, JiT optimizes Latency (Lead Time) while ToC optimizes Throughput.
As the name suggests, Just-in-time is built to deliver the result when promised. If you agreed to deliver a car on Apr 15th, and you know your lead time (say, 14 days), then you know when to start building it (Apr 1st) to deliver just in time for the delivery day.
Theory of Constraints is much simpler. In your production pipeline you have a bottleneck (the slowest step) that allows you to build 100 cars per month. If you want to produce more - concentrate on your bottleneck. There’s a lot of creativity that can go into this:
Your bottleneck must never be idle, otherwise the throughput of the whole system goes down. So you need a buffer - some tasks waiting to be taken by the bottleneck.
Nothing after the bottleneck can break whatever the bottleneck produced. Otherwise your most precious resource was spent for nothing.
Etc, etc, read the books (The Goal, The Goal 2) to learn more.
What’s important is that your bottleneck isn’t your enemy. You don’t necessarily want to speed it up. Instead, you can use it to pace the whole system.
JiT vs ToC in software
The original book “Kanban: Successful Evolutionary Change for Your Technology Business” by David J. Anderson explained the possibilities of using Just-in-time (he called it Kanban by mistake) in software development, it was noted that it’s realistic only for the maintenance projects. The reason is - software development has too much variability, and it’s impossible to make reproducible timelines. If TaskA takes 2 weeks to finish, while TaskB takes 2 months - how can we calculate the Lead Time? We can’t.
But even if we could, Just-in-time is a low-entropy process. Meaning, it requires a lot of effort and discipline to set it up initially, and then to keep it running. Each step has to have just enough capacity and low variability to keep Lead Time stable. Any changes in the staff, client, management or process, and it all quickly goes to nothing. Better to have simple, high entropy (low maintenance) processes that are resilient to changes and don't require careful orchestration.
Theory of Constraints doesn’t use Lead Time, instead it uses throughput as the main metric. It’s much easier to concentrate on the throughput rather than guarantee any specific delivery time. And we don’t really need to measure the Lead Time anyway because most of the time projects have hundreds, thousands of tasks to accomplish. We constantly deliver after each finished task (or a small batch of tasks), gather data and change the directions based on the new information.
Best of the Theory of Constraints
Now that we know that Just-in-time and its Lead Time is an overkill and can concentrate on what’s really important (throughput), how do we implement an actual Theory of Constraints in a software project?
In manufacturing the bottlenecks can be dictated by the physics and logistics of the processes. If our constraint is Step#5 (out of 10), then we have to:
Determine its throughput
Keep the capacity of all steps higher than Step#5 (have redundancy, otherwise other steps will become bottlenecks and you don’t want this chaos)
Guarantee some buffer of tasks before Step#5
And on a daily (or whatever) basis supply the material to accomplish this step. So Step#5 needs to somehow notify Step#4 about its needs.
But in software we don’t have the same limitations. For us it’s easy to identify and change the constraints, which would simplify the process even more. So let’s leverage that. The idea is simple - if we make our 1st step the slowest, then we don’t have to measure or monitor anything. All the subsequent steps work faster than the bottleneck, and so they don’t need additional monitoring.
In practice it means that the Devs need to be the slower than Testing. So Testing should have abundant resources (redundancy). We end up with:
Testing is almost always ahead and can take on new tasks immediately after Devs finish them.
Testing can provide feedback promptly so that Devs don't have a lot of time to switch to the new activities.
Devs are almost never blocked.
We deliver each task when it's ready (or several, depending on the number of people in Testing).
And that’s it! No orchestration needed.
Fitting Business Analysis/UX/Design into Theory of Constraints
Activities related to requirements are usually done either by the Dev team (in which case we just leave the aforementioned process untouched) or by separate people. And if the roles are separated, we now have 3-step process - which one should be the constraint then?
The development part usually is the most expensive. A good ratio of team members is 1 BAs : 6 Devs : 2 Tester. It could be different (and we can also include UX and Design folks), but still the Dev team is usually the largest. So in order to spend money wisely it's better to have Devs as the constraint. Remember, that all non-constraint steps must have redundancy - they have to progress faster. So if we let some other step be the bottleneck, then the Dev team will have spare time, and the project will burn through the budget faster.
If it's not the 1st step that's the constraint, then we have to limit how much work previous steps can do. They can't go too much ahead because this will increase the amount of unfinished work: the tasks the BA described are outdated by the time Devs get to them. We have to update the tasks, which is a waste of the project resources. In ToC terminology this is called inventory cost - the amount of work that's done but not delivered/used, and this is what we want to keep to the minimum.
So in the end what we need is:
BA/UX/Design teams working a little ahead of the development - creating a buffer. A good choice is to have N extra tasks prepared if there are N developers. But be careful if some tasks are too small, in this case we need a little more.
But we don't go beyond the small buffer. If there's nothing to do because all the Devs are busy and the buffer is full, then get some rest. This is one of the hardest/least intuitive part of the ToC - managers don't like when people just chill.
Connecting Theory of Constraints to Continuous Delivery
Continuous Delivery (CD), and Continuous Integration (CI) in particular, postulate that it should be possible to publish the unfinished work. And have other means (feature toggles, branching by abstraction) to hide the work.
In reality even if we hide the new changes, they can still have an impact and break something. At the same time, if we don’t do Continuous Integration, we increase risks too (because of the size of the deliverables), and certainly slow down the development because developers will be blocked while previous work is being tested.
Having development as the constraining step helps with these problems a lot. Because now Devs are blocked rarely. Yet, because I can publish my changes before I fully finish them, we still have situations when Devs need to push to the trunk before previous tasks are tested. Even though this becomes rarer, it does happen.
There are a few remedies that work well for me:
Switching between ToC and CD back and forth. At times the quality seems high and we want to speed things up, other times the quality goes down and we should slow down and become a little more careful.
Dividing the changes into “safe” and “unsafe”. Some changes are certainly safe - documentation, tests. They can be pushed to the trunk right away. Others are certainly not safe and better be postponed until Testing is ready. There’s a huge gray area in between, and a lot of developers struggle to separate safe things from unsafe. But with time and practice (and mistakes) things gradually improve.
Proper branching-by-abstraction, append-only changes (instead of changing a function, copy it and work on the copy, and when the task is finished we do some cleaning up).
So far this has been the best approach that I’ve seen.
Summary
Out of 2 pull systems, Just-in-time seems like an overkill and a high-maintenance process prone to breaking. While Theory of Constraints is a low-maintenance, simple approach, that’s more suitable for software development. In combination with Continuous Delivery it’s probably the best we can achieve, and it’s hard to think of something more efficient and lightweight.
This article discusses 2 modern frameworks of setting up software development processes: Just-in-time and Theory of Constraints. First, we'll discuss how to manage tasks and hand them over between different steps (Business Analysis, Development, Testing) and how/when to release changes to production. Then we'll touch on their relationship with the Continuous Delivery: we'll talk about branching strategies, release readiness and quality considerations.
Pull vs Push systems
Just-in-time (JiT) and Theory of Constraints (ToC) are often mentioned together because both are Pull systems. Meaning that if you have multiple steps in your process (e.g. in software projects we have Dev followed by Testing), the next step (Testing) tells the previous step (Dev) when they’re ready to take on new tasks. Only then Devs can publish their work (e.g. push into the trunk).
This is different from the Push systems - when Devs can publish whenever they finish their job. Push systems are awkward. If Devs submitted 5 tasks into testing, then we can’t deliver the new changes until all 5 tasks are tested. This leads to:
ToC vs JiT in theory
Okay, so both Just-in-time and Theory of Constraints are Pull systems, what’s the difference then? Well, JiT optimizes Latency (Lead Time) while ToC optimizes Throughput.
As the name suggests, Just-in-time is built to deliver the result when promised. If you agreed to deliver a car on Apr 15th, and you know your lead time (say, 14 days), then you know when to start building it (Apr 1st) to deliver just in time for the delivery day.
Theory of Constraints is much simpler. In your production pipeline you have a bottleneck (the slowest step) that allows you to build 100 cars per month. If you want to produce more - concentrate on your bottleneck. There’s a lot of creativity that can go into this:
What’s important is that your bottleneck isn’t your enemy. You don’t necessarily want to speed it up. Instead, you can use it to pace the whole system.
JiT vs ToC in software
The original book “Kanban: Successful Evolutionary Change for Your Technology Business” by David J. Anderson explained the possibilities of using Just-in-time (he called it Kanban by mistake) in software development, it was noted that it’s realistic only for the maintenance projects. The reason is - software development has too much variability, and it’s impossible to make reproducible timelines. If TaskA takes 2 weeks to finish, while TaskB takes 2 months - how can we calculate the Lead Time? We can’t.
But even if we could, Just-in-time is a low-entropy process. Meaning, it requires a lot of effort and discipline to set it up initially, and then to keep it running. Each step has to have just enough capacity and low variability to keep Lead Time stable. Any changes in the staff, client, management or process, and it all quickly goes to nothing. Better to have simple, high entropy (low maintenance) processes that are resilient to changes and don't require careful orchestration.
Theory of Constraints doesn’t use Lead Time, instead it uses throughput as the main metric. It’s much easier to concentrate on the throughput rather than guarantee any specific delivery time. And we don’t really need to measure the Lead Time anyway because most of the time projects have hundreds, thousands of tasks to accomplish. We constantly deliver after each finished task (or a small batch of tasks), gather data and change the directions based on the new information.
Best of the Theory of Constraints
Now that we know that Just-in-time and its Lead Time is an overkill and can concentrate on what’s really important (throughput), how do we implement an actual Theory of Constraints in a software project?
In manufacturing the bottlenecks can be dictated by the physics and logistics of the processes. If our constraint is Step#5 (out of 10), then we have to:
But in software we don’t have the same limitations. For us it’s easy to identify and change the constraints, which would simplify the process even more. So let’s leverage that. The idea is simple - if we make our 1st step the slowest, then we don’t have to measure or monitor anything. All the subsequent steps work faster than the bottleneck, and so they don’t need additional monitoring.
In practice it means that the Devs need to be the slower than Testing. So Testing should have abundant resources (redundancy). We end up with:
And that’s it! No orchestration needed.
Fitting Business Analysis/UX/Design into Theory of Constraints
Activities related to requirements are usually done either by the Dev team (in which case we just leave the aforementioned process untouched) or by separate people. And if the roles are separated, we now have 3-step process - which one should be the constraint then?
The development part usually is the most expensive. A good ratio of team members is 1 BAs : 6 Devs : 2 Tester. It could be different (and we can also include UX and Design folks), but still the Dev team is usually the largest. So in order to spend money wisely it's better to have Devs as the constraint. Remember, that all non-constraint steps must have redundancy - they have to progress faster. So if we let some other step be the bottleneck, then the Dev team will have spare time, and the project will burn through the budget faster.
If it's not the 1st step that's the constraint, then we have to limit how much work previous steps can do. They can't go too much ahead because this will increase the amount of unfinished work: the tasks the BA described are outdated by the time Devs get to them. We have to update the tasks, which is a waste of the project resources. In ToC terminology this is called inventory cost - the amount of work that's done but not delivered/used, and this is what we want to keep to the minimum.
So in the end what we need is:
Connecting Theory of Constraints to Continuous Delivery
Continuous Delivery (CD), and Continuous Integration (CI) in particular, postulate that it should be possible to publish the unfinished work. And have other means (feature toggles, branching by abstraction) to hide the work.
In reality even if we hide the new changes, they can still have an impact and break something. At the same time, if we don’t do Continuous Integration, we increase risks too (because of the size of the deliverables), and certainly slow down the development because developers will be blocked while previous work is being tested.
Having development as the constraining step helps with these problems a lot. Because now Devs are blocked rarely. Yet, because I can publish my changes before I fully finish them, we still have situations when Devs need to push to the trunk before previous tasks are tested. Even though this becomes rarer, it does happen.
There are a few remedies that work well for me:
So far this has been the best approach that I’ve seen.
Summary
Out of 2 pull systems, Just-in-time seems like an overkill and a high-maintenance process prone to breaking. While Theory of Constraints is a low-maintenance, simple approach, that’s more suitable for software development. In combination with Continuous Delivery it’s probably the best we can achieve, and it’s hard to think of something more efficient and lightweight.