Published on: March 3, 2009by Michael Sansolo
There’s an old saying that what gets measured, gets done. But what if there is a problem with the measurement?
Think about all the way we judge whether someone has done a good job and then ask, do we really know? What if the way we measure success somehow warps what we think?
A recent on-line article on hockey, of all things, shed some interesting light on this. Hockey fans know of a statistic called plus/minus. In short, a player gets a plus whenever their team scores when they are on the ice; they get a minus when the other team scores. Seems pretty simple, doesn’t it? By following the stat, you might get a sense of who is playing well.
Or you don’t. In hockey, players rotate on and off the ice constantly with non-stop substitutions. Frequently, a coach will carefully deploy his team to get certain players on the ice in specific situations. For that reason, a good defensive player finds himself on the ice against the opponent’s best offensive players with great frequency. And over the course of the season, that good defensive player ends up with a poor plus/minus stat simply because he always played in the hardest situations. In contrast, a poor defensive player might have better stats because he only gets used in easier situations.
The New York Times Magazine ran an article along a similar vein recently, profiling Shane Battier, a player for the Houston Rockets basketball team. Battier, it seems, has a record of posting modest statistics, except for this: when he plays, his team is always more likely to win because Battier does little things that make that happen. Even when guarding a superstar like Kobe Bryant, Battier finds a way to make Bryant work harder for his points, diminishing his team’s offense in the process.
Like it or not, in business we have the same situation. Frequently, we grade people based on measures of performance that provide an incomplete picture at best and a misleading one at worst.
For instance: manager A runs a store that produces paltry profits year after year. Manager B’s store, in contrast, always racks up splendid profits. However, if manager A’s store has virtually every problem known to exist - a tough neighborhood, an older unit, you name it - it’s possible that manager is actually doing stunningly great work. And, in contrast, manager B might have a store with every possible advantage and actually could produce even stronger profits.
Michael Sansolo can be reached via email at email@example.com .
- KC's View: