You say I’m engaged, I say you’re wasting my time

For content sites, web analysts often look at engagement-related metrics to try to assess whether or not visitors are having a successful visit. After all, there is no transaction like a purchase, to tell us that something “good” happened, if not for our visitor then at least for our business. We may look at metrics like time on site, content pages viewed per visit, the ratio of navigation page views to content page views, and micro-conversions like viewing a print-ready version of an article, or emailing a link to a colleague.

I’ve always suspected there is a fine line between engaging people and wasting their time, especially when you’re dealing with B2B sites. After all, when I’m looking for information as part of my workday, I don’t want to spend a lot of time on a site. I don’t want to view a lot of pages: I want to find the answer to my question right away. Even micro-conversions don’t necessarily mean that my visit was successful: maybe I’m printing out pages or emailing myself a link because I don’t have time to wade through the confusion right now.

I was recently doing an analysis for a site, and was curious about the “engagement” level of two important customer segments. I looked at time on site, content pages per visit, bounce rate, navigation to content ratio (i.e. for a ratio of 2, it would mean that on average, for every content page viewed, visitors must view 2 navigation pages), % visits that contained email to friend actions, and % visits that contained at least one print-ready view. I couldn’t factor  in other potential engagement/loyalty metrics (visit frequency, etc.) because of the large number of shared computers and accounts for this particular site, which is heavily used in a workplace setting. Here’s what I found.

• Segment B had half the bounce rate of Segment A (although both were pretty low).
• Segment B spent 30% more time on the site.
• Segment B viewed 30% more content.
• Segment B viewed print-ready pages in 4% more visits, close enough that I’d consider their usage of this function to be roughly equivalent.
• Segment B needed to go through 8% fewer navigational pages in order to find content.
• Usage of the email function was similar for both segments, just slightly higher for Segment A.

By all measures except the two conversions, I would have considered Segment B to be a good bit more “engaged” than Segment A. Even on the print and email conversions they were roughly the same. But my satisfaction surveys told a different story. I used the responses to our “Were you able to find the information you were looking for?” question to double-check overall satisfaction scores. What did I find?

• Segment B scored 2 points lower than Segment A on overall satisfaction (on a 0-100 scale).
• Segment B respondents were 7% less likely to answer “Yes” to the question about whether they found what they are looking for.

I don’t know if the above differences are statistically significant, but what I absolutely do know is that higher performance on engagement-related metrics did not mean that Segment B customers were happier or more satisfied with the site. If anything, it’s just the opposite.

The best thing you can do for a content site (and other sites, too, IMHO) is to install continuous surveys. That’s the only way you will ever be able to assess the quality of your visitors’ experience online. If you can’t afford one of the for-pay surveys that can be well-customized (from ForeSee Results or iPerceptions), there are still a number of customer satisfaction tools, like 4Q and Kampyle, that allow you to ask your customers straight out whether or not their visit was successful.