After a recent visit to a big online retailers' website, I was invited to take a customer survey when I tried to exit without making a purchase. When given the chance, I'll sometimes take these — not because I love taking surveys, but because the questions brand retailers ask say a lot about what they can't answer through analytics alone. Want to know why your customer are abandoning their shopping carts? So do big-brand retailers, and they get their answers through surveys like these.

But surveys on their own are not always great sources for customer data. While it's great to ask a customer why they abandonned a cart visit , it's pointless to waste a customer's time by asking him questions that are better and more accurately answered through site analytics. 

Survey Post Mortem

In this case the retailer was Banana Republic, one of the Gap Inc. properties. Like many online retailers, Gap has experienced significant growth in online sales (20% in Q2, 2011 to $309 million), but these particular increases only appear to be replacing declining sales at brick and mortar locations. And while 20% growth in online sales sounds impressive, other retailers in the segment are beating that growth substantially. Take, for example, Abercrombie & Fitch, who grew online sales to 132.4 million in Q2 2011,  a growth rate of 41%, according to numbers published by retailsails.com. 

Since all retailers are dealing with issues of higher production prices, lowered consumer confidence and higher shipping fees, I thought the timing of this particular survey was interesting: It happened after leaving a cart without making a purchase, one of the biggest disappointments for online retailers. (According to figures reported by marketwatch.com, shopping cart abandonment rates in the first six months of 2011 are reportedly as high as 75%, ) I immediately assumed the survey would take a hard look at why I left and what Banana Republic could do about it. What I found was a bit different, though.

There are two things to follow while taking a survey: The questions its asks,  and the answers put forward as multiple choice options. Surveys generally have patterns: Pay attention to the number of questions devoted to any one particular topic since these are a decent indicator of the issue's the company is facing behind the scenes. Likewise, the available answers can indicate their best guess as to what the customers might be thinking, or what they actually might want. 

What's On Their Minds

In this case, it was pretty clear what the retailer was thinking about:  Shipping Fees; Site Usability; Competition and Technical Issues, specifically questions about the shopping cart itself. 

First up: Shipping fees and  incentives.

Gap Inc. offers free shipping for orders over $50, but based on the number of questions regarding competitor's fees, they're clearly evaluating what customers are willing to spend for shipping, especially based on order size. (Frankly, it's hard to believe that big retailers are still trying to figure out these numbers. Free shipping, or even membership-based shipping, just plain works. Take Amazon Prime, for example: According to Practical Ecommerce, once an Amazon customer signs up for Prime, they almost immediately spend more money online. Not just a little bit, either: More than twice as much,) 

There were many questions suggesting that customers may be struggling with technical issues: Among the choices to 'Why didn't you make a purchase today?"  were : I had technical problems with the website; and "I had problems during checkout that prevented me from completing my purchase."

In a later question, an answer option suggested that products might be disappearing from shopping carts before the order ca be completed. (What's the cause here? Cookie Duration? Session time-out errors? Server issues?)

Should These Be In A Survey?

Some of the questions were surprising because they could have been better answered through site analytics alone. A prime example: How Did You Get To Banana Republic.com? (They were asking for referrer stats here, one of the most basic analytics fucnitons. There is absolutely no reason this should have been a question.)

These tabs linking Gap Inc. sites probably do more harm than good. Likewise, the survey asked whether I had used navigation tabs that link gap properties.  (These tabs are featured prominently at the top of Gap websites, and  link shoppers between Banana Republic, the Gap, Old Navy and others.) While I understand the urge to link web properties together, there has to be a better way to leverage site visits. After all, if a visitor is about to make a purchase at Banana Republic, why would the retailer remind them that a similar item might be cheaper at one of the brand's other online stores? Conversely, if a visitor is going to make a purchase at Old Navy, having a link at the top of the page 

Even asking whether I used these tabs seems like a waste of a survey question. Did I use them? Who knows? But seeing the number of clicks each tab receives in stats is pretty straight forward.

A Better Approach

If there's a simple rule to customer surveys, it's this: Limit them to perceptual questions. Are visitors leaving because they didn't think the products photos were descriptive enough? Are they leaving because they think (real or not) that you're too expensive? Are they using your site to browse, but planning on buying at your store in person?

Those perceptual questions are worth asking, since it's hard to get meaningful answers through site visit stats alone. Questions about stats, though, should be left out:  While asking a customer a question about why they visited is perfectly legitimate, don't bother asking how long their visit lasted. 

Finally, don't forget the value of A/B testing when you want to test perception. It's fairly straight forward (and actually, critical) to test whether a change in your website has any effect. Then you can gather hard numbers about what your customers are perceiving on line.