The goal

I shouldn't have hesitated so long in reading this. I'm in management of software production activities, and loved Phoenix and unicorn project, and this is a look at the philosophy that drove those philosophies. After this it is so clear why waterfall software production methods are so problematic. It's counter intuitive, until you look at it from a factory perspective. In waterfall you stack a huge amount of inventory up. Stack stack stack. The first boy scout runs down the trail as far as he can go. Maybe all the way to what he thinks is the end of the hike. Then he sits down and waits. Then the next boy gets to walk the trail, the tester, and you're lucky if you only need one go at this, often you need to have three of these folks, each waiting on the previous to get to the end before they start. Then maybe you have an integration step. Maybe this person is fast or maybe they find a bunch of junk that got dropped and some of the other boys have to come back and get it. And then they have to do the walk wait process again from that point.

Only then do you get the last person to walk the trail. Maybe they've been hearing updates all along, telling you when they thought you were going down the wrong path based on what they were hearing. Maybe their walk is the first time they have any insight into the software, er, the trail. It's only when they walk the trail though that they can tell you it's the right way with certainty. Only when they get to the end can they tell you if you stopped at the right point. And only then do you get to confidently prepare for the next day's hike.

And the software customer is a hiker. No software project is just re creating an existing product. Software production is not a normal production line. You create a new thing once - maybe it's just a new version, but you'll only create that once. A production line creates many many copies of one thing. The customer in a normal production context can evaluate a previously finished product. They can determine if they want what you're making, but want it red instead. With software you can only truly evaluate the output from all steps, and only at the end of the process.

The solution is to shorten the path that the first person is allowed to walk before the last person walks. If the first person can only walk as far as the customer can see, now, then the software developer will produce what the customer wants, then the customer will see the next quarter mile, and the developer will produce that.

This requires rapid cycles of development and testing and production, it gets the customer software regularly and asks for their feedback regularly. It keeps that developer actively walking much more regularly, and almost always in the right direction. And often it gives the user something useful very early.

Then you can start to all walk together. You must rope the first boy to the last, so the developer is walking only as fast as the customer. You must limit the release of material (feature needs, bug fixes) to the rate at which the customer can consume it. That looks like continuous deployment, and regular delivery on some short time table. We see it in some companies as many many deployments per day. In my org that is not a good regular end state, but our processes should allow for deployments within hours for emergent situations. And perhaps when you can do that in emergent situations you should just be able to do that any time.