When should marketers truly "change the game?"

I’m currently reading Avinash Kaushik’s EXCELLENT book, “Web Analytics an Hour a Day.” If you’re looking for a book about analytics, this is the place to start.

In the book, on page 255 he relates a story about testing they did for a Flash-based shopping cart. It’s in the chapter about testing (A/B, multivariate, etc.). He says, “…Yet when we tested it live on our website by using an A/B test, the Flash cart initially performed terribly. This was quite a shock. It turns out customers were too used to the more-complicated and ‘cumbersome’ HTML experience and not ready for an optimized cool experience.”

As I read it, this story worked its way into my brain and rattled around a bit. It made me think, “But was it still the wrong idea?”

What I’m getting at is that with anything there is a learning curve. If we see the results of a test like this and say, “Well, they are happier with the more “cumbersome” way of doing things…we’ll just keep it that way,” will we miss an opportunity? I mean, if the Flash-based is TRULY the better way to do it, don’t we do a disservice to the customer by not making the change and providing training? Sure, it may be a difficult transition, but ultimately it will be for good.

I don’t necessarily mean this is the solution for his story because a) Avinash is FAR smarter than I am and b) I’m sure he made the right choice (whatever it was). The story he related was more the catalyst for my thoughts.

My question is, how do we manage the tension between doing what we KNOW will ultimately benefit the customer and what they are comfortable with? If we simply stuck with that which made the customer comfortable, nothing new would ever happen. At what point do we look at the data and say, “Yes, I see what the customer is happier with, but I KNOW that when they learn this new way it will be better for them in the long run.”

I don’t know the answer, but it’s certainly worth pondering.

Technorati Tags: , , ,
When should marketers truly "change the game?"

2 thoughts on “When should marketers truly "change the game?"

  1. The story continues beyond that excerpt (sadly after the book was off to the publisher!).

    We did not take “no” for an answer. We tried again after analyzing the data and simplifying the experience a little bit. This time we integrated surveys between A and B and understood a lot better why people did not like the rich experience (which I was convinced was a better way to shop).

    We learned a lot from the surveys, changed and tried again. This time we beat A by a bit.

    In the last try, created after incorporating VOC, the rich experience beat html becuase it was actually a “dumb rich experience”, really smart under the covers but still easy to use and understand.

    I promise to include the updated story in the next version of the book.

    I am glad that you found the book to be of value, thanks for the feedback.


  2. Avinash:

    Thanks for the comment!

    To be clear, I certainly wasn’t criticizing anything in the book…I’m learning a lot (and running through highlighters!) I’m not even sure that I was referring only to that situation. It just made me think that many people will look at data and give up any thought of change…even when they know the new way is better.

    Thanks for updating me on the rest of the story. Great to hear how it ended up!


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s