Design by Committee Recursion

Trivia question: How many different ways can one answer a true or false question?

Let’s answer this question with a simple thought experiment. You have been hired as the lead architect for the design of a brand new workflow processing solution for Incorporated Dynamic Innovations of Technology, LLC. At your fingertips are all the latest technologies. You waded through weeks of phone interviews to identify the crack team of hackers that will build out the new platform. The first item on the agenda is to capture whether a user has interviewed a subject.

Design problem: Create a form field to indicate whether a subject has been interviewed.

As design problems go, this is a softball. “Ha!” you say. “Easiest problem in the world. I’ll create a checkbox which will indicate whether the interview has happened yet or not. Time for happy hour!” And so, over the course of an overnight marathon of pizza, Mountain Dew, and Nintendo Wii, your crack team writes out your checkbox.

Solution #1: Checkbox: Checked => true, Unchecked => false

A few weeks into testing the analytics design team comes into your office with a problem. They’re having trouble deciding whether an unchecked field means that an interview definitely hasn’t taken place, or whether the user didn’t know if an interview had taken place. You can’t verify that a user has actually thought about the value for the checkbox; whether it’s checked or unchecked, it’s still valid as far as the system is concerned. “Okay no problem,” you say. “Instead of a checkbox, let’s have the user say yes, the interview took place, or no, the interview did not take place. That way, we force our user to think it over and make sure the right data goes in.”

Solution #2: Create a radio button with two values: true and false.

Whew, dodged a bullet there. Now we know for sure, in all cases, whether the interview has taken place! But beneath the surface, a nefarious complication has been introduced. There is now a possible third answer to this question: neither true nor false. Explicitly, there are two options: true and false. But semantically, there is a third implicit option: unknown. Now it turns out that part of the mission of IDIOT LLC’s new platform is to feed data into a warehouse for large scale data mining and analysis. The analytics group is a team of ex-MIT researchers who are well aware of the difference between false and unknown, thank you very much. They want all partnering systems to flag each inbound record with true, false, or unknown, since they can’t be assured that all systems are capturing this data in the same way. So as the architect, you note a business rule that a missing value for our data element represents an unknown. The system interface is turned on, the data starts flowing, and everybody’s happy.

Skip ahead a year or two. Organization wide, IDIOT LLC has had its share of low profile project failures and perhaps one or two high profile ones. Meetings have been held amongst the executive leadership to identify ways to improve data quality. In particular, there seems to be an unexplained source of error in the analytic reports. Our humble data element is captured by several systems, which feed into the warehouse. Some systems require a true or false value, and some do not. But all systems report a missing value as unknown. The root cause of our data quality problem seems to be that unknown includes cases where a lot of users simply forgot to add the value. The CIO stands up and says, “This is going to come to a halt, gosh darn it.” All users will now have to explicitly state, in all partnering systems, if the interview has occurred, if the interview has not occurred, and if the user simply doesn’t know. As the architect, you grudgingly accept the order from on high, and modify your solution accordingly:

Solution #3: Drop-down box to capture true, false, or unknown

But now we’ve come back to the same problem we started out with. There is a semantic difference between someone expressing unknown as a value (explicitly unknown), and someone opting not to provide a response (implicitly unknown). With three explicit values, we’ve implicitly created a fourth value for our humble true/false question: implicitly unknown. So we have three explicit values (true, false, and unknown) and a brand new implicit value (implicitly unknown).

But this is absurd, right? I mean, there isn’t a real difference between “unknown” and “forgot-to-answer,” is there? Well to a human being, not really. But to a computer, or a large, interconnected network of systems, there is a very real difference. And indeed, eventually, this new implicit option can be encoded into data exchange formats as well. A colleague mentioned to me yesterday a new requirement from a customer that four values for a true/false question be captured in a report: true, false, explicitly unknown, and implicitly unknown!

We started out with a simple data element, true or false. We went off the reservation once we started trying to guess the user’s intent through a combination of what the user did and did not do. Is the statement false? Does she really not know, or is she expressing that she is certain it’s false? It’s not unlike a fourteen-year old girl trying to guess whether the cute boy in her biology class likes her by just how brusquely he pushed past her in the lunch line (actual reason: french bread pizza day! Sweet!). The other side effect is that with each new recursive slice we add into our true/false data element, we dilute the value of the original data element. (How much could it possibly matter whether something is explicitly or implicitly unknown? In either case, you still don’t know!!!)

The correct answer to my original question, of course, is that there are infinitely many ways to answer a true or false question, just as there are infinitely many numbers between 0 and 1. Only a human being can tell where the line in the sand belongs; the committee will never get to that point.