Process is a lie. All of it.

If you’re spitting and fuming because you believe in software development processes, bear with me. I have a point to make, and you’ll enjoy it.

If you’ve never believed in software development processes and you’re cheering me on, take a seat and a deep breath. I have a point to make, and you’re not gonna like it. But you need to hear it.

Process is a Lie?

Process is a lie, because process is a plan for how we’ll work. And plans are lies. All of them.

A plan says, “This is what will happen, and this is what we’ll do.” A more realistic plan says, “This is what will happen, and this is what we’ll do. These are things that could happen instead, and this is how we’ll handle them.” And a truly realistic plan says, “This is what will happen, and this is what we’ll do. These are things that could happen instead, and this is how we’ll handle them. And in case something happens that we haven’t foreseen, these are the resources we keep in reserve and the options we keep open.”

And even truly realistic plans fail. Nineteenth century German Field Marshall Helmuth Graf von Moltke told us: “No battle plan survives contact with the enemy.”

Some developers and teams are process averse. And one common reason is that they’ve experienced plenty of process failures.

And yet Field Marshall von Moltke made plenty of battle plans. He wasn’t saying “Plans are bad.” He was saying “Plans are never enough. But they’re still essential.”

Process is never enough. But it’s still essential. Process failure might indicate a flaw in that particular process; but often, it indicates that no process survives contact with the enemy.

Now the enemy in this case is not the customer (at least, not if your customer relations are healthy). The problem itself is the enemy, and your team is the invading army, trying to find its weak points, pick it apart, and conquer it. And the problem fights back! It hides minor details until they become critical. It changes due to unforeseen circumstances. It’s a lot bigger than it looks. And sometimes it draws a lot of its problem friends into the battle when you least expect them.

Your process is your battle plan; and it is wrong, in ways you can’t foresee until they happen. But that doesn’t make it wrong.

Wrong, but Not Wrong?

OK, now Martin’s getting confusing: it’s wrong, but it’s not wrong. What gives? Well, a plan can be (will be) wrong in details and yet still be right overall. Like all plans, your process is what you expect to do, not what you actually end up doing. A process or plan establishes a basic framework for how you’ll handle the simple, mundane matters that arise; but realism tells us that things are going to happen that aren’t covered by the plan. The process guides you in handling what you can foresee, so that you can spare more attention and thought for the eventualities you can’t foresee.

The plan or process lets you handle simple stuff without thinking, so you have more thought to devote to the hard stuff.

So a well-designed process is good, as far as it goes; but you have to be ready to step outside it or beyond it when the need arises. And a really good process includes a process for changing or sidestepping the process. The U.S. Constitution includes an amendment process for this reason.

One of the most common sources of process disillusionment is the ISO-9000 family of standards for quality management systems. Many an organization has implemented failed ISO-9000 initiatives; and while I won’t name names to protect the not-quite-innocent, more than one organization has implemented “pretend” ISO-9000 initiatives to attain certification, but never really follow their own standards.

And yet ISO-9000 is, at its core, a good process for designing processes. It really amounts to this:

1. Document what your process is.
2. Document how you gather data and measure it.
3. Document how you correct it when it fails.
4. Document how you change it when needed.
5. Document how you handle exceptions when needed.
6. Document how you’ll train people in the process.
7. Document that you’re following the process, including data.
8. Document that you’re correcting when needed.
9. Document the exceptions when they happen.
10. Document training results.
11. Document changes to your process.

Forgive me if I’m being dense, but what could be wrong with any of those items?

No, when ISO-9000 fails, it’s almost never because of ISO-9000 itself. Rather, it’s because your process stinks; and when you’re process stinks, either your team abandons it, or they blindly follow it. In some cases, the stinky process comes from outside “experts” or consultants who don’t know how your team actually works. Often it’s a “canned” process that the team is “shoehorned” into, with disastrous results.

What’s missing, then, is an often-overlooked step 0: Decide what your process really is. Don’t let some outsider tell your team their process; ask your team what their process is. Ask them questions and get them thinking about it. And then document it, document your data, etc., etc., etc.

Yes, it’s possible that your team needs to learn and adopt better processes. Absolutely. But the wrong process can be worse than no process at all. If your team isn’t ready or able to be circus horses, then a circus horse process will fail. You can cajole and fume and order and threaten all you want, and it won’t change anything: some teams aren’t ready or suited for some levels of formality. Or informality: some people adapt well to more formal processes, and find informal processes to be chaotic time-wasters. They’re proud circus horses, not smelly, dirty cattle.

CAFÉ, or CAFÉ? Your Choice…

So yes, you need a development process, whether you think you do or not. In fact, if you don’t think you have a process, you’re probably following the most common development process, CAFÉ:

Figure 5: CAFÉ – Code And Fix, Endlessly

And if you’re really lucky, sometimes you hit the end of that process. Maybe. If you define “Bugs” in a very forgiving fashion.

The trick is finding the right process, one that fits well how you’re already working. Then try to describe that process so that you and your team can reach a common process. Then document that process. Then apply that process, and see where it falls short, and make it better over time.

Most popular software development processes today are incremental, building small chunks of functionality and then adding more over time. Why? Because that way you never get too far down a wrong path. You get quick feedback on what you’re doing right and wrong. And then you fix the code that’s wrong. (In fact, these incremental processes are all, at their core, more structured descendants of CAFÉ.)

Well, developing your development process should also be incremental. Don’t try to change everything overnight. Do try to change. Then look at what worked, and do more of it. Look at what didn’t work, and fix it. Repeat, repeat, repeat. When it comes to your process, you want the other CAFÉ: Change And Fix, Endlessly.

Short Cut (n): Something that will take longer and cost more, but might gain some immediate goal. (Odds of gaining that immediate goal average around 7.3%.)

Example: “Yeah, we know we needed to test that, but it would take all week. We need to ship today, so we’re going to take this short cut instead: we’ll test after we ship.”

Commentary: Really, if the short cut were a good idea, why didn’t you plan to do it that way in the first place? As Professor Tolkien wrote, “Short cuts make long delays.” Or as I keep telling people: Short cuts take longer and cost more, every single time.

Show Stopper (n): The largest bug that isn’t fixed yet.

Example: “Last week, the show stopper was that the brakes cut out. This week, the show stopper is that the buttons are cyan instead of teal.”

What would you get if you married Ambrose Bierce’s Devil’s Dictionary…

CYNIC, n.

A blackguard whose faulty vision sees things as they are, not as they ought to be. Hence the custom among the Scythians of plucking out a cynic’s eyes to improve his vision.

…with the engineering adage Murphy’s Law…

If there’s more than one possible outcome of a job or task, and one of those outcomes will result in disaster or an undesirable consequence, then somebody will do it that way.

…?

Murphy’s Dictionary: an ongoing series of definitions of what words really mean on a typical development project.