The myth of data driven decisions.

In a previous blog post “Rolling them down the hill” I was making the point that people’s decision making tends follow a path of an instantaneously formed opinion based on aesthetics followed by a process of looking for evidence to support this initial choice. In retrospect I may have over simplified it but there is still something there that I believe is largely true.

What made me think of it again was this quote that came up in one of my RSS feeds:

“When faced with two choices, simply toss a coin. It works not because it settles the question for you, but because, in that brief moment when the coin is in the air, you suddenly know what you are hoping for.” Minimal

There has been a lot of work done in the field of decision making from examining established techniques to complicated medical investigations looking at things like the “ventromedial prefrontal cortex” and its probably worth mentioning that I have studied none of it. Even so I have a theory that much of the practical techniques are pretty much for show and that ultimately, good decisions makers rely on intuition & courage more than anything else.  This is quite similar to what Malcolm Gladwell was writing about in Blink but the distinction I am making is that the process of decision making is a delaying tactic while the decision maker tries to work out how they “feel” about it in order to choose. They will do what feels right.

Fork in the road by MikeSchinkel, on Flickr

I know that  people like to think they have an open mind and gather evidence before making a decision but in my experience the data collected rarely gives a  purely positive or purely negative result. You are going to get pros an cons either way and ultimately you are going to have to make a “feel” call in the end.

“In the end” means you run out of time and are forced to make a call theoretically based on on what ever data you have gathered to that point. Your decision that may be better because you have had some time to think about it a little more or read something that changed your perspective slightly. But not really evidence.

In agile software development there is a saying that the best possible outcome is making the right decision and the worst possible outcome is not making a decision. The wrong decisions sits somewhere between the two. The sooner you make a wrong decision the sooner you discover that you where wrong. If you don’t make a decision then you are not making progress at all, you have stopped moving and probably stalled everyone working with you. You make not realised this at first but eventually you will find your loyal troops have moved on to other work.

What this means to me is that to get the best out of our teams and ourselves its probably best to stop pretending that we go through a decision making process and have more courage to make intuitive decisions.