Online measurement: Are we there yet?

Another thought-provoking piece in Business Week, this time on the state of the online measurement business and the trouble of getting consistent and accurate numbers.

As I attended Aggregators Upsetters last night (more on that soon), the topic of aggregation is top-of-mind this morning, so the article was very timely.

As long as I've been in the business, there are some things that have always been true about online measurement, tracking and reporting:

1) It is easy for owners to get precise numbers about traffic, visits, members, sales data and so on. Good internal reporting systems deliver incredibly accurate numbers down to the place, pence and person.

2) You can always tell smart businesses from, er, those "in need of some help" by how good they are with thier data. Tracking and reporting is a great place to pass judgement.

3) There are a range of reporting options depending on budget and need. Examples range from simple free open-source solutions (AWStats), to slightly more complex (but still free) systems integrating ad tracking and reporting (Google Analytics, previously Urchin), to more complex and in-depth analytical tools (MMA's services, including Digital ROI).

4) External partners have always been able to deliver real numbers. That is true for clickthroughs, sales data, ad and pageview impressions, sign-ups and registrations. Depending on the deal, rich demographic and psychographic data has also been highly accurately reported.

5) I have never seen or worked with an aggregator (ComScore/Media Metrix, Nielsen//NetRatings, Alexa) that has accuratetly mirrored internal data.

6) Of all the aggregators, anyone who puts even a grain of sand's worth of trust in Alexa's numbers is a complete fool. They're a joke.

7) The numbers from the aggregators are generally higher than internal numbers. As a marketer, they always make you feel good compared to reality, don't they?

8) Businesses hate sharing real/internal numbers, because of the variance between internal numbers and what gets reported by the aggregators. This is especially true for start-ups. They are worried how they will compare to the "unreliable" numbers from competitors and aggregators.

Anyone have 9 and 10 to round out the list? ;-)

Here's my favourite (and controversial) quote from Web Numbers: What's Real?:

The dirty little secret of Silicon Valley is that no one knows exactly who is going where on the Web. That flies in the face of the impression that online advertising is the most dependably trackable ad medium of all time, a big reason spending on Web ads is expected to grow 33% this year, to $16 billion. But confusion over traffic measurement could cast a chill over the Web 2.0 craze. Valuations for startups such as Facebook Inc. and YouTube Inc. appear to be doubling every few months, but those numbers are based on traffic figures that could be misleading.

Max Kalehoff calls the article "overly sensational and overplayed". What do you think?

By the way, if I didn't know better I'd say the folks at Business Week are trying to make a real go at reporting on that there innerneter thingy, albeit from a slightly skeptical and headline-grabbing vantage point. Damn those pro journos. Need more? See their CEO Guide to Social Networks.

Discussion: IP Democracy, AttentionMax, VCMike's Blog and Slashdot.

Technorati Tags: aggregators, measurement, startups, business.week