Stop Watches and Yardsticks

Recently, I've had occasion to reflect upon one of the initiatives at work. Managers would like to measure what the workers do. Its a two-edged sword. On the one hand, there's the Big Brother aspect that we've been assured is not the case. On the other hand, you can't improve if you don't measure.

I build software. If, instead, I were to dig ditches, we could easily measure the volume of dirt displaced. But what metrics does one apply to the construction of software? Lines of code? Bug Reports? Cyclometric complexity statistics?

Last Tuesday, John Cunningham of Band XI spoke at the West Michigan XP group and described some of his frustrations doing XP within IBM. He would faithfully deliver the numbers to his higher ups that they requested and know they were utterly misleading. I knew exactly what he meant, because I've done the same.

You record the time spent doing tasks and what gets lost is the utter non-value spent in some of those tasks and the absolute priceless-ness of chance encounters that come and go in two minutes. The former are reported to management and the latter are not. They serve to push management a step away from reality.

In short, I object to measurements that mislead, or can be gamed. Once I understood this I sought a metaphor that I could use to communicate what I have in mind. Consider a track meet, what measurement tools are used there? Some officials carry around stop watches and other officials carry tape measures.

In particular, consider the high jump competition. You set the bar a measured distance above the ground and the best athlete is the one who doesn't knock it down. You measure this performance with a tape measure.

But suppose an official who's been watching footraces all day comes along with stop watch and no tape measure. He's perplexed until he notes that the guy who jumps highest also stays off the ground longest. He decides to measure the athlete's performance with just a stop watch.

What would happen then? The athletes would jump differently. But would they jump higher? Subtly, the fact that the measurement has changed will change the way the athletes jump. I don't think those changes will result in higher jumps. Performance suffers due to using the wrong measurement.

It seems silly to measure the high jump this way, but suppose you had lots of cheap stop watches and only a few expensive yardsticks. Or suppose you don't understand the field of endeavor well enough to see how yardsticks are better than stop watches.

Conversely, it'd be silly to take a yardstick or tape measure to a footrace.

So, I'm not saying "don't measure." I'm saying that you have to understand what you're doing well enough to select the correct measurement.

And you have to use, and not misuse the correct measurement.

Let's go back to the Big Brother consideration I touched on earlier. In the UK there is now one video surveillance camera for every 14 people. Those cameras are in place to "catch" wrongdoing. Contrast this with a training context where your coach videotapes your jump to analyze your form, a one frame at a time. He's got a purpose that you and he agree upon. He's not out to "catch" anything except those things you want to correct in order to jump higher. He's your Coach, not your Policeman.

Do you see a cop with a radar gun? Slide into something's radar shadow and decelerate. Then look at your speedometer. You change your behavior so he won't "catch" you speeding even when you're doing nothing wrong.

The relationship between Big Brother and those under his thumb is adversarial, whereas the relationship between Coach and athlete is cooperative. The fact that "Big Brother" even comes to mind speaks volumes of the culture of an organization.

Measurement in an adversarial context is a negative-sum game. In this context, everyone is encouraged by the system to replace reality (that might cast me or my political allies in a negative light) with whatever figures we can spin into a Potemkin village filled with smiling happy peasants, each tugging on their forelocks as the Empress sails by.

The cultural question probably hinges upon a question to the measurement taker that John Cunningham raised last Tuesday: "Do I want to know what's going on, or do I want to impose my will upon the situation?"