Whacky Web Metrics
by Jerry Hart
Having a website is no longer a competitive advantage. Managers are expected to quantify the impact of their web properties on the bottom line of their businesses. As a result, the web analytics industry is booming and is predicted to grow to $3 billion by 2010. In my view, web analytics will never be an exact science, even though its influence in the decision-making process is growing. Issues inherent in web data analysis make it difficult to get accurate, insightful data. This article’s purpose is to make you aware of the pitfalls associated with web data analysis so you can plan for or avoid them. Here are five ways your data can be skewed by the very technology you seek to leverage.
1. AOL Proxy Servers
AOL proxy servers are a killer to traditional web site analytics programs. When an AOL user requests multiple documents for multiple such as webpages, each request may come from a different proxy server or a different IP address. Since one proxy server can have multiple members going to one site, webmasters should not make assumptions about the relationship between members and proxy servers when designing their website.
Implication for your business:
If one AOL user views 10 pages on your site, your website analytics tool could be misled into thinking that 10 different users came to your site and each viewed only one page. I know that when I see a lot of single page loads to the homepage, I start considering making minor changes to entice people to click further. Incorrect data can lead you to make incorrect decisions. If you do not account for or fix the AOL proxy server issue, the reports that quantify the number of visitors (or unique users) have the potential to be highly inaccurate. Although many corporations and some ISPs use proxy servers, they do not pose a problem because the number of users coming through non-AOL proxy servers to websites is small. But because AOL drives up to 50 percent of the traffic on some sites that I analyze, it presents a huge problem.
2. Random Spiders
A spider is an automated program designed to gather information on webpages. Most log file analysis tools recognize when Googlebot (Google), Scooter (AltaVista), Slurp (Inktomi) or any of the major search engine spiders visit a site. They know that the visitor viewing webpages is actually an automated program and is not a legitimate user. Typically, most decent analysis tools automatically filter out data resulting from spiders.
This is good news. Using a tool that automatically recognizes and filters out automated spiders helps you get closer to reporting, analyzing, and making decisions on the correct data. Here’s the bad news: Any programmer can create a spider and send it to your site. There are thousands of unknown spiders, and some of them are crawling your site, inflating your web data even as you read this chapter. These unknown spiders usually get through the average web analytics tool filters because those spiders don’t identify themselves as such, instead appearing as regular users. E-mail harvesters are an example of random spiders. Ever wonder how a spammer got your e-mail address? A popular method is the e-mail harvester, an automated program built to traverse the web for e-mail addresses so that they can be added a database.
Implication for your business:
What are the repercussions to your decision-making processes when a spider hits your site 1,000 times in 30 seconds and doesn’t get filtered? Well, think about it, if you just started an ad campaign or new marketing initiative, and suddenly a significant amount of traffic came to your site, you might attribute this success to your new ad campaign or marketing initiative. In actuality, the increases you saw in your data may have been the result of an unknown and unfiltered spider. Even worse, if you assumed that your campaign was a success, you might extend the campaign and spend more of your budget on it, essentially throwing money out the window.
If your web analytics solution does not require reading log files but instead uses a small piece of JavaScript on each page, then you may not have this issue. Spiders typically can’t read JavaScript and will not register in your website analysis reports.
Some software uses logic to automatically filter out automated spiders. If a user loads a given amount of pages in a certain period of time, then the user can be automatically filtered out. SurfAid software from IBM also keeps track of the growing list of spiders and updates the software to filter them out. This is the first program I have seen that recognizes the importance of automatically filtering out suspicious activity that can lead to highly inaccurate data.
3. Frames
If your site is developed in frames, take your number of page loads and divide that number by three. That is how many pages may actually have been loaded on your site. A framed site typically loads three pages for every single page a user views on their screen. Frames tackle many problems with site development, but open up a slew of other issues with tracking and marketing your website.
Implication for your business:
Guess what the above scenario can do to your data…triple it! If your site (or a part of it) is developed in frames, then any data you have reported for the site (or a part of it) may be tripled.
4. Flash and Dynamic Sites
Flash is becoming increasingly popular as a website development tool. You might notice on one of these sites that as you move around the site, the URL bar (where you type in web addresses) never changes.
Implication for your business:
Although the site is beautiful on the surface, developing a site in this manner will devastate your web analytics initiatives. It will appear to your analytics tool that the entire site is composed of only one page. No matter how many different pages a user views, it will always appear as if the homepage is being loaded over and over again. Fortunately, not all Flash and dynamic sites are programmed in this manner, but many still are. For those that are, true analysis can be difficult and sometimes impossible. Analyzing data for conversion metrics, return on investment (ROI) for various marketing campaigns, top entry points
into your site, user paths through site, fall off rates and many other essential internet business metrics are not usually possible without paying for additional programming changes to correct the one-page site dilemma.
5. Sharing Secure Certificates
When a user leaves the public area of your site and moves to a secure area (maybe where a credit card is processed), sometimes a very different, unique URL is used. When secure transactions happen on someone else’s server where you are “sharing” a secure certificate with others, you do not have access to that log data.
Implication for your business:
Once visitors start to buy a product or complete an application/lead, and at some point leave your site to go to the shared secure site, their activity starts getting tracked on that shared secure site. That means they will essentially “disappear” at some point in your site’s log, giving you an incomplete view of user activity through one of the most important parts of your site: the conversion. Getting this data from a shared hosting environment could prove costly, and may even be impossible, depending on how inflexible your website host is.
Realizing that your web data is going to be flawed, and understanding the technologies and how they can affect your analysis, is half the battle. Here are six ways you can “fix” web data problems.
1. Use cookies or unique-user logins
Use cookies. They help recognize true repeat visitors and give marketers a much better handle on where a user’s path begins and ends, how many pages a user viewed in a particular session, and so on. Using cookies with your analysis tool may take some time to configure. But stick with it. Their use has the alternative benefit of alleviating the proxy server issue. I feel much more confidant in web data when cookie technology is used in conjunction with a web analytics tool. You can even go one step further and
use unique-user logins, but unfortunately this isn’t always feasible. However, if you are analyzing the effectiveness of an intranet site where users have to log in to get access, unique-user logins will help you get data that is even more accurate than when you use cookies. Remember: Some cookies will be rejected or deleted occasionally by cookie washers (software designed to clean all cookies off of a computer). But the vast majority won’t be, so your data should be pretty solid.
2. Educate yourself on the basics of programming
The accuracy of your data hinges on how your site is programmed. Learn just enough about web programming to be able to articulate to your programmers what the pitfalls are. Your being able to “talk the talk” will help your programmers to avoid some of the issues that come up with dynamic and framed websites. If your site uses Flash technology, explain to your designer that each time a link is clicked, the URL bar should change. You don’t need to understand how this gets done, but you do need to be able to articulate these kinds of changes in a language that your programmers and designers will understand.
3. Analyze your data for anomalies
Sort your page views in ascending order. If you see a user loading 21,000 pages in one hour, you may want to filter out that user, or that user’s IP, because it is probably an automated spider doing the loading. A human being is not likely to be able to load and read 350 pages per minute.
4. Focus on what matters
Conversions, e-mail newsletter signups, brand exposure, whatever matters to your business, is what you analyze, period. Create a list of priorities and work on getting the most accurate data for the statistics that are most influential to your decision-making process.
5. Audit log file data with custom solutions
Get your programmers to build an administration function that aggregates the number of times a thank you page was loaded, for example, and sort that report by day, month and year. Such a tool will be a tremendous asset for you to audit your web conversion stats. Having a tool to aggregate a basic (but highly important) statistic for auditing purposes is very helpful in ensuring that the data you have is accurate. It also ensures that the methodologies used to get that data are sound and able to be replicated.
6. Get an experienced Internet metrics analyst
If all this is too much detail for you, but you still see the value in having highly accurate web data, you may need to hire someone whose job is to fully understand your business and determine how to use web metrics to gauge success in achieving business goals. Ideally, the person in charge of analytics needs to be part programmer, part analyst and part marketer rolled into one:
- The programmer understands what technologies can be implemented to provide accurate data that the business feels comfortable using to make decisions. This person fully understands the implications of website design on web analysis.
- The analyst understands the needs of the business and what data the business needs to prove what’s working and what isn’t. This person likely sits in on high-level business meetings and will help develop some of the business strategy.
- The marketer is the one who understands the customers and how to entice them to come to the site and, more importantly, how to get them to take the desired course of action.
# # #
Jerry Hart is CEO of Hart Creative Marketing and an Associate Partner of Marketing Operations Partners.
To find out more about Marketing Operations Partners' measurement-related services , please call 408-243-7881 or e-mail sales@mopartners.com.
