Google Running Secret Quality Labs?

By Nathan Weinberg

Henk Van Ess has started off a new blog, Search Bistro, with a scoop he claims to have found: Google’s evaluation labs. Henk says that one of his students at the University of Amsterdam was a participant in the Google Rater Hub, which pays people to rate the accuracy of Google search results in order to improve results and reduce spam. Henk links to a job posting looking for a “Quality Rater” for Google, mentioning the job tasks as “”help with work on a search quality evaluation” and “Candidates will evaluate search results and rate their relevance”. Henk claims that the site is at eval.google.com (which is 404 if you aren’t in the program) and has posted a Flash movie of it in action.

Now, while the story seems legit, a lot of red flags pop up. The story was immediately posted at Slashdot by the blogger (and posted in the third person) and Henk has been emailing it to various bloggers, myself included. His “tone” is suspicious as well, with a lot of self-congratulatory excitement over being Slashdotted. Of course, if I got Slashdotted on my very first blog post, I’d probably be a giddy schoolgirl as well. Besides, even if its a new blog, Henk isn’t exactly an unknown property, having been a newspaper and website editor, and a sometimes contributer to Search Engine Guide.

Also, its not like its an unbelievable story. Yahoo had paid “Surfers” (the actual job title) to do the same thing in the past, and most multi-billion dollar companies engage in some sort of quality assurance and focus grouping. My “odd feeling” probably has more to do with indigestion than intuition.

The real question is: Who in the hell wouldn’t want this job? Search, give a thumbs or down, repeat until you get a check. Where can I sign up?

UPDATE: Google Blogoscoped confirms the reports. However, Philipp notes that the screenshots are outdated, and the raters have no influence on PageRank (their results are only used for internal purposes).

Posted:
June 2, 2005 by Nathan Weinberg in:
-->

8 Responses to “Google Running Secret Quality Labs?”

  1. Bart N. Says:

    Has anyone found the signup page yet ?!! :)

  2. SEOToolSet Blog Says:

    Secrets revealed

    What is eval.google.com? The mystery subdomain has been puzzling webmasters and SEOs alike since its discovery. Well Henk Van Ess claims to have the answer. What’s the secret of Eval? Soylent Google is people. That’s right. According to Van Ess’ new…

  3. SEOToolSet Blog Says:

    Secrets revealed

    What is eval.google.com? The mystery subdomain has been puzzling webmasters and SEOs alike since its discovery. Well Henk Van Ess claims to have the answer. What’s the secret of Eval? Soylent Google is people. That’s right. According to Van Ess’ new…

  4. Henk van Ess Says:

    Dear Nathan,

    Thanks! I badly needed that lesson in humbleness from you. You are one of my favourite webloggers so any attention will do. Yes, I did post my weblog on Slashdot myself. My self-congratulatory excitement is over, but Slashdot refuses to withdraw my posting! I tried to explain to the New York Times that I’m only blogging to get publicity, but they refused to believe me. A German weblogger says I did it all for the hits, but only my hoster agrees (they like the extra $$). An Australian site thinks I am postponing oblivion by posting information in several takes, but my seven months old son disagrees since he loves to be feeded as much as possible.

    Seriously, let’s concentrate on the content, not on who I am. Will try to post some more details, f.e. Google’s spam policy, the impact of the lab and if possible some comments from Debbie Frost from Google.

  5. Dan Neal Says:

    I applied for the job in january. Didn’t get it though. It was offering a rate of £10/hr. Gutted!

  6. pigfoot’s weblog » Google Running Secret Quality Labs? Says:

    […] bsp;

    Slashdot 的 Google’s Secret Lab, a Flash movie is here. InsideGoogle 的 Google Running Secret Quality Labs? 大意� […]

  7. Not a Googler Says:

    Okay, maybe I’m an evil corporate stooge, but it bothers me when people so flagrantly violate their NDAs. Have people no sense of honor and integrity?

  8. Andy Says:

    Hmmm. I think that this NDA was always going to be violated.

    Expecting people on £10 an hour to keep their mouths shut, when the info they have could be so interesting, is a bit unrealistic I think.

Leave a Reply