The ongoing discussion of the ways you can stop search engines from indexing specified pages and the use of the noindex meta tag was the topic of Matt Cutts blog the other day. Matt started a poll of what people would like the noindex tag to do: A: Don't show the page at all; B. Find some middle ground; or C:Show a link to the page.

The results have been massively in favor of not showing the page at all, 617 to 61 and 53.

Interestingly Matt argued that Google needs some discretion of what pages the noindex tag stops from being listed in the search results. He uses a couple of instances of governemnt essential sites that have been dropped because they mistakenly used that tag.

"The vast majority of webmasters who use NOINDEX do so deliberately and use the meta tag correctly (e.g. for parked domains that they don’t want to show up in Google). Users are most discouraged when they search for a well-known site and can’t find it. What if Google treated NOINDEX differently if the site was well-known? For example, if the site was in the Open Directory, then show a reference to the page even if the site used the NOINDEX meta tag. Otherwise, don’t show the site at all. The majority of webmasters could remove their site from Google, but Google would still return higher-profile sites when users searched for them," Matt blogged.

The post lays out the difficulties Google encounters when errors by site owners use the tags incorrectly and stop peopple from accessing information that should be available to the web.

The comments have a solid cross section of well known marketers from our industry and present some good insights.

Joost de Volk, search strategist for Onetomarket notes "The fact that some websites get the noindex wrong by accident is a problem, I can understand, but you don’t solve a problem a minority of websites has by forcing a majority of people to change their ways. You solve that problem by educating the people maintaining those website."

Philipp Lenssen of Blogscoped added: "I think it’s a pretty clear-defined case: webmasters put “noindex” in their page because they don’t want the page indexed or shown. As you can’t know whether a webmaster perhaps accidentally put the noindex there, you have to err on the safe side and do what you’re told. Or how would you feel if people started to interpret Google terms of services in terms of, “oh maybe their lawyers just misspelled this and really mean something else, I’ll ignore it.”

Also, please do not try to push webmasters to always use a Google tool — like a URL removal tool — to do stuff; while Google search is close to a monopoly there are still other engines out there, and webmasters have better things to do than toggle a dozen tool’s configurations."



Posted on Tue, 26 Feb 2008 11:28:46 -0500 at http://feeds.searchenginewatch.com/~r/se...226-112846