Measuring Success

In some professions, the metrics for success are straightforward. A surgeon performs an operation – does the patient live or die? An auto mechanic repairs a transmission – can the customer drive their car home? A boxer lands punches – does the opponent get up from the canvas before the referee can count to ten? An actress auditions for a part in a movie – does she appear on the big screen come opening weekend?

When I narrow in on the software industry, there are some jobs that are pretty clear on success metrics as well. A sales rep has a quota to hit every month, every quarter, every year. A programmer has to write code that meets the product requirements he was given, and the finished product has to meet quality standards.

Not everyone in the software industry has such clear cut measures of success, though. I’m one of those people, where my job is hard to describe, even to my own co-workers, and my managers don’t always have a clear picture of what I’m doing from day to day. A lot of the time, even I don’t have a clear picture of what I’m doing from day to day, and there isn’t a specific end goal in mind. I’m the only person in my company doing my role, which makes it difficult to compare myself against other employees.

Every quarter, I come up with somewhat arbitrary objectives for myself that I propose to my management as the basis for my bonus pay. There may be some minor tweaking of those objectives by my management – dial this up a notch, dial this other one down a notch, replace this third one with another comparable one that they had in mind, etc.

Invariably, three months later, when it comes time to review the past quarter’s objectives and my attainment of those objectives, we have to fiddle with the results. Instead of doing A, B, and C, I wound up doing A, C, and D. D is comparable to C, so it’s all good. And I overachieved on B by 50%, so it’s fine that I missed the mark on A by 10%.

And so it goes, every three months. Today was one of those days when I met with my boss about my bonus objectives and setting the next measures of success.

Coincidentally, when I was driving home from work this evening, I was listening to sports radio talk about the baseball Hall of Fame results that were announced today. It struck me that the Hall of Fame voting this year illustrated that even baseball has difficulty with measuring success.

I have never used steroids, period.

This was the year when a pitcher (Bert Blyleven) made the Hall of Fame without meeting the previously-held standard metrics for success (less than 300 wins, no Cy Young Awards); and, a hitter (Rafael Palmeiro) was left out of the Hall of Fame despite exceeding traditional benchmarks that guaranteed inclusion (more than 3,000 hits AND more than 500 home runs).

The baseball writers who vote on the Hall of Fame ballots reviewed these two players’ performances and allowed for a fudge factor in the measures of success. For Blyleven, they were able to look past the “easy” numbers and realize that 287 wins for some pretty bad teams is close enough, when you add in 3701 strikeouts and a whole slew of advanced statistics that I don’t pretend to understand. For Palmeiro, the voters ignored the “easy” metrics, instead focusing on the positive test for steroids that earned Palmeiro a suspension – no matter how much finger-wagging he did before Congress.

A salesman can do whatever it takes to make the deal, taking shady orders, just so long as he meets quota. A programmer can cut corners to ship the code on time, sacrificing quality and fault tolerance, skimping on documentation, half-finishing features.

Sometimes you simply have to look past the established measures of success in order to evaluate performance.


  1. When I as the whole web server / proxy engineering team for a few months when Sun and AOL split up iPlanet, I reported to Brian Ahern. I hardly even talked to him, he was concerned with App Server all the time. So, one day I asked him, “I really haven’t gotten much feedback from you, how do you think I am doing?” His reply was, “Major customers call all wound up about the web server. I give them your number, I have never heard one complain a second time. To me, that means you must be doing great.”
    There’s your measure, how many unhappy customers do you make happy (not go away, because making customers go away is bad). Start with 100%, and every customer the complains / escalates about the same problem (if it is in your control to fix), is a downward tick on your bonus.

    Oh yeah, and the Hall of Fame is useless if Pete Rose is not in there!!!!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s