Based on a study of 7,500 Web sites conducted by Penn State University, Google emerged as Webmasters favorite. C. Lee Giles, the David Reese Professor of Information Sciences and Technology who led the research team which developed a new search engine—BotSeer—for the study, found the result "surprising".

Giles said, "We expected that robots.txt files would treat all search engines equally or maybe disfavor certain obnoxious bots, so we were surprised to discover a strong correlation between the robots favored and the search engines’ market share".

Some of the results are:

More than any other search engine, Google's is most favored and called most often.

Take a look at the chart below and guess who the prodigal one is.

Apparently, the study is titles, "Determining Bias to Search Engines from Robots.txt (PDF)," Barry Schwartz has a local copy for easy download.

Click here to subscribe to our RSS feed to get a daily digest of news around search engine industry. PageTraffic SEO Blog is updated four times a day and is ranked as one of the best search engine resources blog by Pandia!