I want to be a robot

[lovingboth]

Recently I saw something in some weekly/monthly computer magazine about a site which will take a URL and go off and get it for you whilst pretending to be Google's spider, googlebot.

Its name was something like becomethebot.co.uk, but that's not it.

Any one seen this?

Comments

[ciphergoth.livejournal.com]

Don't know, but this might be helpful: a Firefox plugin that achieves the same thing.

https://addons.mozilla.org/en-US/firefox/addon/59

[lovingboth]

I've seen that one - it doesn't have pretending to be the various bots included by default (just IE, Opera, and Mozilla) plus it doesn't do the IP spoofing that some sites need in order to be fooled.

I saw something about this site in one of the weekly magazines about a fortnight ago.

[skibbley.livejournal.com]

A defence is here: http://googlewebmastercentral.blogspot.com/2006/09/how-to-verify-googlebot.html

[lovingboth]

I don't want to defend against it, I want to do it :)