An attempt to trade quality for cost or schedule actually results in increased cost and a longer schedule.

Steve McConnell,
Professional Software Development

What has long been known in other businesses is true for software development as well: if you cut corners for shorter schedules or lower costs, you will get longer schedules, higher costs, and higher defect rates; but if you take the right measures to lower defect rates, you can get lower defect rates and shorter schedules and lower costs. As Crosby wrote, “Quality is free.” But it’s free only in terms of ROI, meaning the investment must be made first; and it’s only free if you first define what you mean by “quality”.

Fortunately, Crosby provided the appropriate definition as well: quality is conformance to requirements. That can be a concrete, quantifiable definition; but in some way it just moves the problem down the road, leaving us to define requirements: not just the term, but the specific requirements of our projects. It leaves us with this inescapable truth:

If we don’t know our requirements, we can’t have quality.

…we must define quality as “conformance to requirements” if we are to manage it.

Philip B. Crosby,
Quality is Free: The Art of Making Quality Certain

Poorly defined requirements are endemic to the software development industry. Boehm’s research on factors that affect development schedules and costs show that:

Excellent requirements analysts can reduce a project’s schedule by almost 30%, while inadequate analysis can increase the schedule by over 40%.

Again and again, schedule pressures lead teams to start developing before they have sufficient requirements definition.

Even though requirements analysis is a key skill, the topic isn’t as “hot” as new technologies and tools that are promoted by vendors and conferences and magazines. And many development teams feel swamped just trying to keep up with technologies and tools.

Martin L. Shoemaker

Data from Boehm et al., Software Cost Estimation with COCOMO II

Among all personnel factors, Analyst capability has the widest range of impact (multiplier range of 2, worst case divided by best case). Teams may tout their application experience as a strength; but application experience has an Influence Range of only 1.51. Application and platform experience combined have an Influence Range of 2.11. Teams would never throw out their domain knowledge and develop for an entirely new platform; but poor requirements practices have almost the same Impact Range.

These teams aren’t foolish, yet they foolishly let a critical aspect of their process get out of control on project after project. A look at their team rosters may give a clue as to why: while there are many roles on the rosters, there may be none with requirements as a primary responsibility. Marketing and sales have requirements responsibilities, but many conflicting responsibilities as well. Lead engineers are supposed to verify requirements; but they are also too busy, and are commonly focused on solutions, not requirements. Designers and developers also focus more on how than on what. Traditionally in software development, analysts have primary responsibility for and are evaluated on the correctness of requirements.

The role of requirements analysts is
to define the problem in a verifiable form,
so that teams may recognize a valid solution.

And next you must ask: who owns that responsibility in your organization? If the answer is “no one” or “I don’t know”, there’s a ripe opportunity to cut your schedules by 30% to maybe even 50%, all while improving your quality.

Code is not enough. It’s all about requirements; and that’s all about communication.

A business classic tells us that Quality Is Free. The title is intentionally provocative: no, quality isn’t free, it just pays for itself. But first, you have to pay for it.

And that, unfortunately, is where we fail in the quality game so often. Corporations seem addicted to the practice of compartmentalized budgeting, or what I think of as “bucket budgeting”: you’ve got a bunch of different buckets you pour money into at the start of the fiscal period; and each bucket can only be spent on a particular purpose. When that bucket is empty, you can’t accomplish that purpose any more. Oh, the company may have some reserve you can tap; but you’re going to get frowned at for exhausting your budget.

I understand bucket budgeting as a planning tool. I think it makes perfect sense, for the same reason you should make estimates out of small elements. In fact, it should the exact same reason, because these budgets should be estimates, not shackles. You’ll correct over time.

And that’s where the failure comes in. Somehow, some way, in too many organizations, those buckets are shackles. Your bucket is what you can spend, what you will spend — regardless of the bottom-line impact of your spending. Even if every $1 spent out of your bucket brings $1.20 into the company, you only get to spend what’s in your bucket. This isn’t a new complaint, of course; and smart managers certainly keep an eye out for ways that spending money can save or generate more than they spend. But less bold managers don’t like to rock boats. They live within their buckets, because overspending their bucket gets them a bad review. It takes courage to stand up and make a case for more money in your bucket, unless you have a very clear, simple chain between the money you spend and the money that comes back in.

And the quality equation is particularly susceptible to bucket budget shackles. Quality does pay for itself, but it seldom shows up anywhere near the buckets where the costs came from. The cost of quality is measured in extra labor and time up front on preventing defects, along with extra labor and on the back end detecting and correcting defects. It’s also training time, which takes time away from “productive” work. It’s also management time and communications effort in getting everyone — execs, workers, and customers — to understand that seemingly slower time is actually faster in the long run.

The benefits of quality, meanwhile, are in reduced support costs, reduced rework costs, and increased customer satisfaction and loyalty. These do affect the bottom line; but they don’t put money in the buckets of the managers who have to pay for the quality in the first place.

A common sarcastic reaction I here among the workforce is “Quality is free, so management thinks they can have it without paying for it.” And sadly, this reaction is often justified. But I don’t think most managers are really that clueless. I do think many managers are shackled by bucket budgeting, and unwilling to buck the system for something that won’t have an effect on their budgets. The effect may be the same as cluelessness; but if we don’t understand the proper cause, we can’t devise the proper correction.

And no, I don’t know what that correction is. I mean, to me, the answer is simple: start treating budgets as estimates, not shackles; but I don’t think little old me is going to change corporate culture that drastically just by saying so.

Final note: this isn’t inspired by anything at my current client. Really, it’s not: I’ve been complaining about bucket budgeting for over a decade. But it’s true that my client is currently entering a phase where they’re trying to invest in quality in pursuit of benefits in the long run, and I want to do my part for that. There are forces that will push against that effort, and forces that will push for it. I’m doing a little writing to help me clarify how I can help push in the right direction.

Note to doctors: No, I don’t really mean that title. I use provocative titles to get attention and capture an attitude. What? What are you doing? You’re going to stick that thermometer where?

Doctors are the stupidest users. If you’ve ever had to write software for doctors, you’ve discovered this: the phrase “RTFM” was made for doctors. They just can’t be bothered to read even the simplest help docs. They can’t bother to learn even simple tools that a bright grade-schooler can master.

OK, that’s the programmer perspective. Now let’s look at it from the doctor’s perspective. On her desk is a small mountain of medical journals she needs to read to keep up with her specialty.

Next to those is a small mountain of textbooks for the new specialty she’s trying to learn.

Next to those is a small mountain of new regulations and guidelines she must comply with to maintain her license.

Next to those is a small mountain of insurance guidelines she’ll probably never have time to read but should if she wants to make sure she’s charging within guidelines.

Next to those is a small mountain of insurance paperwork that demands her scarce attention. She has no time for it; but if she doesn’t keep up with it, patients may be denied treatment that would’ve been approved if she had.

Next to those is a small mountain of accounting statements that she’d rather not bother with, but has to if she wants to pay her student loans and malpractice insurance.

And next to all of those is a large mountain of patient histories, test reports, specialist reports, and hospital reports she needs to keep straight in order to treat her patients.

And somewhere, buried under all of those papers, are a few pictures of the family she hopes to see once or twice this week.

And along comes this programmer person who says, “What’s the deal? This is easy! Just read this, and this, and then try this, and learn this, and look at this, and it’s easy! You’re a doctor. You’re supposed to be smart. This should be easy!”

After she schedules the programmer for an emergency barium enema, the doctor goes back to her work.

This is part of the reason (by no means the entire reason) that there’s a profession of Medical Technician. These people’s responsibility, in part, is to be the doctor’s user interface to programs that the doctor just has no time to master. They have time to specialize in arcane software. They usually have more technical experience.

But really: should it be this way? Not that I want to put Medical Technicians out of work; but shouldn’t the programmers spend more effort understanding the doctors and their needs, rather than requiring the doctors to understand programs and programmers?

For years, I’ve spoken and taught about a principle I call Conservation of Complexity. For any given task, there’s a minimum complexity required for that class. You can’t make it less complex. (But trust me, some idiot can always make it more complex…) If I’m automating that complexity, I take some of that work onto myself, and leave some for the user; but no matter what I do, I can’t reduce the complexity below the minimum. And being lazy by nature, I’ll want to do the minimum necessary to meet the requirements: minimum complexity for me, and let the user do the work.

But we can cheat! We can’t reduce the complexity of one instance of the task; but we can reduce the complexity of multiple instances, especially when those instances are performed by multiple users. In so doing, we can reduce the net complexity in the system.

Let’s say the user and I split the complexity, C: I get 0.5 C, and he gets 0.5 C. Now if 100 users do the task 100 times, we have:

Cnet = 0.5 C + 100 * 100 * 0.5 C = 5000.5 C

Cave = Cnet / n = 5000.5 C / 10000 = 0.50005 C

So we’ve cut the complexity in half. That’s great, right? That’s why we write code, right?

But suppose some idiot — i.e., me — didn’t make that app very usable for the user. Oh, it’s easier than working by hand, or I won’t sell any copies; but say it’s 0.8 C. In that case:

Cave = 0.80005 C

So I took an easy way out. The system is still less complex on the whole. We’re still winning, right?

But now, let’s go the opposite way. Let’s say I put in the extra effort to reduce the user’s work to 0.2 C. And let’s assume this takes me a simply ghastly effort, 2 C. That means that:

Cave = 0.2002 C

By me working twice as hard, the system as a whole works roughly 80% less.

Now these numbers are just examples, of course. But I think they make my point: if all I worry about is getting code out the door, I may not work hard; but my users have to work a lot harder. Too often, we programmers see our perspective too clearly, and the other guy’s perspective too dimly. (In other words, we’re human.) But the smarter thing to do is to put forth the extra effort to reduce the user’s complexity. Because after all, there’s money to be made in reducing your user’s workload!

I’m reading Alan Cooper’s The Inmates are Running the Asylum; and with practically every page, I’m reminded of Conservation of Complexity. And of the doctors.

Also see Joel Spolsky’s discussion on mental models. He approaches the same idea: the programmer has a mental model of how the code works; the user has a mental model of how the code works; and great software results when the programmer’s mental model is very close to the user’s. It’s easier for a team of motivated programmers to change their handful of mental models than to change all those users.

On a mailing list where I hang out, a participant recently said (paraphrased): “He believes that popularity proves quality. I believe that there is almost no correlation between quality and popularity.”

We hear this sort of thing all the time. There’s an implication among self-appointed elites that “the masses” — i.e., everyone who’s not them — just can’t recognize quality. It’s assumed that “popular” is proof that something is bad. You see this attitude in film snobs who insist that an Oscar nomination for The Return of the King is some sort of travesty, because the film is a popular fantasy and not some art house flick or some historical epic. To be fair: you see it in Lord of the Rings fans who for years have been telling others who didn’t like the books that they just didn’t appreciate great literature. And before the films, you could see it among the literati who snubbed Lord of the Rings because it’s a popular fantasy rather than a dreary, post-modern, self-referential, obscurantist yawn. You see it in opera buffs who assume the rest of us are subintelligent because we don’t share their passion for opera. You see it in young rebels who look down on the lives of the conformist “sheeple” and who demonstrate that they are individuals and not “sheeple” — by all dressing and talking and acting and piercing alike. And you even see it in gourmets who extol the virtues of French food over more pedestrian fare like food from McDonald’s.

But the truth is: they’re wrong, every single one of them. They proceed from two clearly false assumptions: that there is one clear, objective, inarguable standard of quality; and that of all human beings, they somehow have been born with/been granted/achieved the unique ability to pronounce what the standard is.

But the fact is just the opposite. If I can avoid butchering the Latin too poorly, de gustibus non disputandum: with taste there can be no dispute. Or in the modern vernacular: there’s no accounting for taste. When someone tries to tell you that his tastes are objectively correct, he’s demonstrating how self-centered he is or how shallow his thinking is.

Does that mean there are no things that are objectively better than some other things? Can’t we all agree that Shakespeare is better than “The Simpsons”? Nope: I could gather up quite a debate on both sides of that issue; and the pro-Simpsons side would be every bit as educated and erudite as the pro-Shakespeare side.

Can’t we all agree that French food is better than McDonald’s? No, for multiple reasons: many people dislike new tastes, and prefer comfort and familiarity; not everyone likes the spices in French food; and if you grew up with French food every day, you might see it as “normal” and McDonald’s as a new experience, where novelty makes it attractive.

And so on, and so on, and so on. If you take any “objective” measure of general quality (as opposed to quality for a particular purpose, which may be assessed much more precisely) and examine it all the way down to its roots, you find personal tastes, past experiences, biases, and other responses that aren’t objective at all. There’s no objective measure of quality.

Except one. See, like many things that are immeasurable in the small, quality is measurable in the large, through statistics. No one person can absolutely proclaim that a certain thing is a quality product; but we can measure with reasonable precision how many people accept and endorse the quality of a product, by virtue of their purchases. In other words, the list writer I paraphrased has it exactly wrong: the closest thing we have to an objective measure of quality is popularity. If a significant number of people enjoy a product, then the odds that you will like it are higher. We’re all individuals, not ruled by statistics; but statistics are a useful piece of information to help you find products to try. Quantity purchased is a valid measure of quality. The market identifies products that the largest number of people accept as quality products.

And before anyone chimes in about betamax, QWERTY keyboards, CDs vs. albums, Microsoft, or any other oft-cited “evidence” that the market can produce the “wrong” answer: go reread my post, because you still missed the point. Don’t force me to go haul out the evidence that shows the conventional wisdom is wrong in every one of these cases, or I’ll produce so much it crashes the server.

Which column describes your project management techniques?

Disclaimer: This has nothing to do with any of my current projects or clients. The person who inspired this already knows who he is, and is firmly in column 1. Any other resemblance to actual persons or events is purely coincidental.

It might be you.

Process is a lie. All of it.

If you’re spitting and fuming because you believe in software development processes, bear with me. I have a point to make, and you’ll enjoy it.

If you’ve never believed in software development processes and you’re cheering me on, take a seat and a deep breath. I have a point to make, and you’re not gonna like it. But you need to hear it.

Process is a Lie?

Process is a lie, because process is a plan for how we’ll work. And plans are lies. All of them.

A plan says, “This is what will happen, and this is what we’ll do.” A more realistic plan says, “This is what will happen, and this is what we’ll do. These are things that could happen instead, and this is how we’ll handle them.” And a truly realistic plan says, “This is what will happen, and this is what we’ll do. These are things that could happen instead, and this is how we’ll handle them. And in case something happens that we haven’t foreseen, these are the resources we keep in reserve and the options we keep open.”

And even truly realistic plans fail. Nineteenth century German Field Marshall Helmuth Graf von Moltke told us: “No battle plan survives contact with the enemy.”

Some developers and teams are process averse. And one common reason is that they’ve experienced plenty of process failures.

And yet Field Marshall von Moltke made plenty of battle plans. He wasn’t saying “Plans are bad.” He was saying “Plans are never enough. But they’re still essential.”

Process is never enough. But it’s still essential. Process failure might indicate a flaw in that particular process; but often, it indicates that no process survives contact with the enemy.

Now the enemy in this case is not the customer (at least, not if your customer relations are healthy). The problem itself is the enemy, and your team is the invading army, trying to find its weak points, pick it apart, and conquer it. And the problem fights back! It hides minor details until they become critical. It changes due to unforeseen circumstances. It’s a lot bigger than it looks. And sometimes it draws a lot of its problem friends into the battle when you least expect them.

Your process is your battle plan; and it is wrong, in ways you can’t foresee until they happen. But that doesn’t make it wrong.

Wrong, but Not Wrong?

OK, now Martin’s getting confusing: it’s wrong, but it’s not wrong. What gives? Well, a plan can be (will be) wrong in details and yet still be right overall. Like all plans, your process is what you expect to do, not what you actually end up doing. A process or plan establishes a basic framework for how you’ll handle the simple, mundane matters that arise; but realism tells us that things are going to happen that aren’t covered by the plan. The process guides you in handling what you can foresee, so that you can spare more attention and thought for the eventualities you can’t foresee.

The plan or process lets you handle simple stuff without thinking, so you have more thought to devote to the hard stuff.

So a well-designed process is good, as far as it goes; but you have to be ready to step outside it or beyond it when the need arises. And a really good process includes a process for changing or sidestepping the process. The U.S. Constitution includes an amendment process for this reason.

One of the most common sources of process disillusionment is the ISO-9000 family of standards for quality management systems. Many an organization has implemented failed ISO-9000 initiatives; and while I won’t name names to protect the not-quite-innocent, more than one organization has implemented “pretend” ISO-9000 initiatives to attain certification, but never really follow their own standards.

And yet ISO-9000 is, at its core, a good process for designing processes. It really amounts to this:

1. Document what your process is.
2. Document how you gather data and measure it.
3. Document how you correct it when it fails.
4. Document how you change it when needed.
5. Document how you handle exceptions when needed.
6. Document how you’ll train people in the process.
7. Document that you’re following the process, including data.
8. Document that you’re correcting when needed.
9. Document the exceptions when they happen.
10. Document training results.
11. Document changes to your process.

Forgive me if I’m being dense, but what could be wrong with any of those items?

No, when ISO-9000 fails, it’s almost never because of ISO-9000 itself. Rather, it’s because your process stinks; and when you’re process stinks, either your team abandons it, or they blindly follow it. In some cases, the stinky process comes from outside “experts” or consultants who don’t know how your team actually works. Often it’s a “canned” process that the team is “shoehorned” into, with disastrous results.

What’s missing, then, is an often-overlooked step 0: Decide what your process really is. Don’t let some outsider tell your team their process; ask your team what their process is. Ask them questions and get them thinking about it. And then document it, document your data, etc., etc., etc.

Yes, it’s possible that your team needs to learn and adopt better processes. Absolutely. But the wrong process can be worse than no process at all. If your team isn’t ready or able to be circus horses, then a circus horse process will fail. You can cajole and fume and order and threaten all you want, and it won’t change anything: some teams aren’t ready or suited for some levels of formality. Or informality: some people adapt well to more formal processes, and find informal processes to be chaotic time-wasters. They’re proud circus horses, not smelly, dirty cattle.

CAFÉ, or CAFÉ? Your Choice…

So yes, you need a development process, whether you think you do or not. In fact, if you don’t think you have a process, you’re probably following the most common development process, CAFÉ:

Figure 5: CAFÉ – Code And Fix, Endlessly

And if you’re really lucky, sometimes you hit the end of that process. Maybe. If you define “Bugs” in a very forgiving fashion.

The trick is finding the right process, one that fits well how you’re already working. Then try to describe that process so that you and your team can reach a common process. Then document that process. Then apply that process, and see where it falls short, and make it better over time.

Most popular software development processes today are incremental, building small chunks of functionality and then adding more over time. Why? Because that way you never get too far down a wrong path. You get quick feedback on what you’re doing right and wrong. And then you fix the code that’s wrong. (In fact, these incremental processes are all, at their core, more structured descendants of CAFÉ.)

Well, developing your development process should also be incremental. Don’t try to change everything overnight. Do try to change. Then look at what worked, and do more of it. Look at what didn’t work, and fix it. Repeat, repeat, repeat. When it comes to your process, you want the other CAFÉ: Change And Fix, Endlessly.

Short Cut (n): Something that will take longer and cost more, but might gain some immediate goal. (Odds of gaining that immediate goal average around 7.3%.)

Example: “Yeah, we know we needed to test that, but it would take all week. We need to ship today, so we’re going to take this short cut instead: we’ll test after we ship.”

Commentary: Really, if the short cut were a good idea, why didn’t you plan to do it that way in the first place? As Professor Tolkien wrote, “Short cuts make long delays.” Or as I keep telling people: Short cuts take longer and cost more, every single time.

Show Stopper (n): The largest bug that isn’t fixed yet.

Example: “Last week, the show stopper was that the brakes cut out. This week, the show stopper is that the buttons are cyan instead of teal.”