Can product management be scientific?

I know a spell
That would make you well
Write about love, it could be in any tense, but it must make sense

Belle & Sebastian – Write About Love

Some companies believe that product management can be reduced to scientific experiments.Instead of using intuition and customers interaction one should run experiments and measure results.

While I have great faith in measurable product management , I think that the dream of product management without the human factor is wrong and dangerous.

Everyone seriously involved in pattern recognition and data mining knows that one can’t just throw tons of raw data into an algorithm and expect to gain (artificial) intelligence.

In most cases it is hard to build a large data set to train the algorithm. Once such data set is built , the raw size is too big for any algorithm to train on. As a result ,the raw data needs to be reduced through feature extraction. For example, if we want to build a face recognition algorithm in a video stream we can help the algorithm by removing the soundtrack. While in theory the soundtrack can add information to the algorithm, we guess it is not very helpful.

The process of feature selection and even dataset selection involved intuition and domain knowledge. This is similar to the generic scientific model.

In 2008 wired magazine claimed scientific method is obsolete  in “The End of Theory: The Data Deluge Makes the Scientific Method Obsolete“. The article claims that models are not needed anymore, as data is stronger than models.

However, even the first example is wrong

Google’s founding philosophy is that we don’t know why this page is better than that one: If the statistics of incoming links say it is, that’s good enough

But Google’s early  success was  not just because of the algorithm. The clean UI,text only ads and great performance were crucial. I’m confident it was intuition\product management that led into these decisions. Moreover, the statistics for incoming links  from fraud (link farms) are also very high. The algorithms needs “help” on the features that identify fraud.

Google Suggest brings another example. It is a great feature which interactively “guesses” the search term for the end-user.

For example, typing the word “Robert” suggests the following :

Google Suggest For Robert

Google Suggest For Robert

But looking for the word “Naked” brings no results at all:

Google Suggest For Naked

Google Suggest For Naked

But “Nak”still shows some alternatives:

Google Suggest For Nak

Google Suggest For Nak

Does anyone think the algorithm decided on this feature based on statistics 🙂 ?

Obviously, someone decided that following the real statistics of the human mind would be too dangerous.

I’m very much in flavor of usability research and detailed numerical specs. But in most scenarios, the psychology, human interaction and models are crucial for a building a great product.


Tags: , , , , , , ,

3 Responses to “Can product management be scientific?”

  1. Uri Says:

    you are baiting me.. 🙂

    The short response is that without supporting data, intuition is very dangerous. There are many tools to test innovation safely ( A/B testing is just the tip of the iceberg ).

    without data, we would be following the one whose intuition shouts the loudest (decibels rise according to management rank). This has created many great successes, but an order of magnitude more failures.

    The best path: cogitate, innovate, and than use the scientific method. And ALWAYS: speak with your customers, meet them and strive to establish a relationship.

    • ophirk Says:

      I agree. Intuition should lead to experiments measured by results and iterated very quickly.
      My main point was the raw data alone without model or customer interaction is not enough.

      P.S. – Experiments can be expensive (e.g. Windows Vista,Google Buzz) or complex(different pricing schemes) but surveys and usability studies can help a lot.

  2. Uri Ar Says:

    Love this post.

    Science is a great tool. When it comes to designing products, it plays an important role. However, if there was a scientific method of creating the ‘perfect’ product – either there would be no competition or the market would equalize itself.

    As for the article – Facebook are giving Google a nice lesson about the limits of science.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: