The Art of Web Analytics

Traditionally, usability is tested qualitatively by conducting interviews with a small number of potential users.

The interviews take place in a “lab” where the participants are given certain tasks to complete on the web site in question. While interacting with the site, a moderator observes their behavior and asks them to share their experiences.

While I’m a big fan of this kind of usability test, the method also has a number of drawbacks. The most important being that it takes a long time to carry out and doesn’t provide the kind of instant and continuous feedback we know from web analytics.

As such, I find it interesting to search for possible ways of using web analytics to measure usability. What we need, ideally, is some kind of easy-to-understand, yet robust, web metric that indicates how easy it is for the visitors to navigate our site.

Such web metric should not necessarily replace qualitative tests in a lab. Rather, it should help us to obtain early warnings and point to the parts of our website which are most in need of usability attention.

Moving beyond standard metrics
Measuring usability by means of web analytics is not as easy as one should perhaps think.

Traditional web metrics, to be sure, will not do the job: The number of page views per visit, the bounce rate, the number of visits per visitor, the average visit duration, etc. None of these metrics can tell us anything about navigational problems.

Although it is true that frustrated or confused visitors might end up producing many page views per visit, for example, highly engaged visitors might do the exact same.

Introducing Back Navigation Rate
So, the question remains: How can we identify usability problems simply by looking at click stream data? Is it possible at all?

Some years ago, my colleagues and I at Netminers set out to answer that question. We came up with the idea that if there is any way to identify frustrated or confused visitors it must be based on how often they go back and forth between the same pages.

A visitor who runs into a problem, we thought, is likely to stop and go back to make sure he or she didn’t miss something. This might even happen repeatedly since visitors are likely to double-check before abandoning the site altogether.

We therefore decided to develop a technique for tracking what we call “back navigations” and “turn pages”. Consider the following click stream.

The blue dots represent pages seen by a single visitor. The arrows indicate the direction of the click stream.

A is the entry page, while B and C are pages that lead to a particular interest point, D. Having looked at D, the visitor decides to go back to C, and further back to B. Finally the visitor makes a new move forward from B to E.

As can be seen from the figure, there are four forward navigations and two back navigations. We say in this case that the Back Navigation Rate is 50% (2 divided by 4). Notice also that D can be seen as a turning point in that the visitor decides to turn around here. We call D a “Turn Page”.

When is a high Back Navigation Rate bad?
Is it good or bad that the Back Navigation Rate is 50%? Well, it depends.

You cannot always say it’s a bad thing, because sometimes visitors just like to explore links by moving back and forth on the site. This is especially the case on the home page where visitors often explore many different links before selecting a specific page for closer inspection.

According to Netminers Index (i.e. a series of benchmarks based on all the data we collect across all of our clients), the Back Navigation Rate is in average 30%. Back navigations are therefore not as rare as you might first think.

Keep it low for your check-out procedure!
However, imagine now that you are looking exclusively at a check-out procedure or a similar conversion process. In this case you certainly do not want users to go back 30% of the times! A check-out procedure, to be sure, is supposed to be linear, not zigzag, and, in our experience, here the Back Navigation Rate definitely shouldn’t exceed 5%.

Turn Page Rate
But you can do more to identify problematic back-and-forth movements. Notice that in the click stream presented above D appear as a turn pages. This, however, happens only once.

Consider now a case where the visitor produces repeated turn pages.

In this case B and C are both turn pages, and both of them appear as such two times. We can capture the difference between the former and the latter click stream by what we in Netminers call Turn Page Rate. This rate is calculated as follows: First, you slice your data so that you only look at turn pages. Then you divide the number of page views by visits. This rate is 1 for D in the former click stream , but 2 for both B and C in the latter.

What do you think?
What is your opinion about Back Navigation Rate and Turn Page Rate? Can you think of any flaws of these metrics? Do you have suggestions for some alternative and perhaps better usability metrics? Please let me know!