The above image shows some recent data I pulled from Google Webmaster Tools (GWT). We don't rely on GWT data for reporting, but regularly refer to it internally to analyze trends. But I'm becoming skeptical about the reliability of the data even for that very limited purpose.
You see, I know that there are clicks for these queries, I clicked on them myself... Several times...
Furthermore, some of these queries are actually showing up as reported keywords in Google Analytics. Now that's ironic...
Maybe Google is doing some weird filtering that I don't understand. Fine. But it doesn't make the data any more useful to me.
This isn't the first time someone has complained about the accuracy of Google Webmaster Tools data and I'm sure it won't be the last. Nonetheless, it creates a huge problem for people, like us, who try to help our clients understand their sites' visibility in Google's search results.
Here are a few quotes from search people who I have a lot of respect for:
The short version, if you want to skip me ranting like a lunatic: Google Webmaster Tools query data is, as far as I can tell, completely, 100% useless. It’s not a good ‘relative comparison.’ It’s so wrong that it might actually be a bad idea to use it at all.
I agree with +Ian Lurie and don't use the GWT data when I'm doing any sort of analysis. There are some fundamental issues with how the data is presented (UX issues which could lead people to wrong conclusions), gathered (sampled) and processed (rounded).
- AJ Kohn (Blind Five Year Old)
My message to everyone: think about data quality and validity before using Google Webmaster Tool data for your research, predictions or reporting purposes. It is ok to analyse trends and movements, using individual data points is not recommended.
- Jan-Willem Bobbink (NotProvided.eu)
Let me show you an example of where this Webmaster Tools impression data is misleading and can actually be harmful to your attempts to manage client expectations.
Some have defended the value of keyword data in WMT, while others have called it flat out useless. I’m in the nearly useless camp, as my own internal research leads me to believe that I’m missing around 89% of the organic search queries that I used to get. Furthermore, the click data for the few keywords Google is providing me appears to be off by a factor of +/- 400% due to how data is being estimated/rounded. Finally, you can only go back for 3 months, so it’s clearly not ideal.
Ultimately, I recognize that:
The only truth is this: No method of tracking, and no form of analytics are accurate. The best they give is a snapshot of some random stuff that may or may not be indicative, and is a bit like 2 or 3 different scouts in positions that give limited but different perspectives passing chinese whispers of what they have seen.
- Black Knight (Cre8asiteforums.com)
And I agree with Richard Baxter that:
you should always perform your own analysis, on your own sites to come to your own conclusions.
And so, here are some defenses of GWT data:
The GWT vs GA Disclaimer from Google:
Webmaster Tools data may differ from the data displayed in other tools, such as Google Analytics. Possible reasons for this include:
Webmaster Tools does some additional data processing—for example, to handle duplicate content and visits from robots—that may cause your stats to differ from stats listed in other sources.
Some tools, such as Google Analytics, track traffic only from users who have enabled JavaScript in their browser.
Google Analytics tracks visits only to pages which include the correctly configured Analytics Javascript code. If pages on the site don't have the code, Analytics will not track visits to those pages. Visits to pages without the Analytics tracking code will, however, be tracked in Webmaster Tools if users reach them via search results or if Google crawls or otherwise discovers them.
Some tools define "keywords" differently. For example:
The Keywords page in Webmaster Tools displays the most significant words Google found on your site.
The Keywords tool in Google Adwords displays the total number of user queries for that keyword across the web.
Analytics uses the term "keywords" to describe both search engine queries and AdWords paid keywords.
The Webmaster Tools Search Queries page lists shows the total number of keyword search queries in which your page's listing was seen in search results, and this is a smaller number.
There may be a lag between when the numbers are calculated and when they are visible to webmasters—although data gets published in intervals, we are continually collecting it.
If you can no longer see a search query you saw recently on the Webmaster Tools Search Queries page, make sure you haven't filtered the results by country or type of search.
Webmaster Tools aggregates query information, and displays search queries once the count of each query reaches a certain threshold. Your logs may show a particular query as having a high rank for a certain day or period, but that query does not appear in on the Search Queries page. If the query continues to be a top referrer, however, it will move to the top of our aggregate results and will appear on the Search Queries page.
Also, Webmaster Tools stats show only search queries from Google. Your log files may combine search results from all search engines.
From Google's John Mueller:
With more companies and products using Google Webmaster Tool search query data, and Googlers Maile Ohye and John Mueller vouching for the accuracy of the numbers, it's a shame so many people have dismissed free insight as invalid. Don't be one of them! Downloading, categorizing, and trending over time is the best way to get the most out of what Google is giving us.
- Ben Goodsell (Search Engine Watch)
As SEOs, we should be using every tool at our disposal to evaluate how effective our strategies are. It’s true that Webmaster Tools has clear limitations. However, if you approach it with an understanding of those limitations, it can provide valuable insights into keyword data lost to [not provided]. In other words, looking through a paper towel tube is still better than being blind.
As for me, I remain skeptical and cynical. Even if GWT is "accurate" based upon very specific definitions (I'm even skeptical of that), it still remains rather useless in terms of understanding how Google users are clicking on your pages. At the end of the day, it's reporting 0s where I KNOW that clicks occurred.
Bottom Line: I recommend that you don't bet your reputation on the usefulness of GWT data.
