Snapbot

1. Snapbot

   #1

   The snapbot spider is a nuisance. It is registered as a user and not as a spider.
   
   Here are relevant debates about snapbot:
   http://www.webmasterworld.com/forum11/3268-1-30.htm
   http://www.webmasterworld.com/forum39/4267.htm
   http://www.cre8asiteforums.com/forums/index.php?showtopic=37374
   

2. Re: Snapbot

   #2

   So ... Not to sound all computer-illiterate and such, but what ARE these spiders, anywho? First guess is a program sent by search engines to find searched-after things, yes? no?

3. Re: Snapbot

   #3

   Yep. The search engines use these "spiders" (computer programs scanning websites) to enter information about the web into their databases. This way, if the spider finds the string "ShadowEagle" on this site, the search engines will make your "ShadowEagle" search hit on this site.

4. Re: Snapbot

   #4

   Aah, Got it. We learn something new every day, hm? And so what i gather from the third link, the snapbot is gathering entire website data instead of just single strings of information, thus sucking up bandwidth, yes? Or something like that

5. Re: Snapbot

   #5

   Yes. And it doesn't follow "bot rules". It is a bad bot.

6. Re: Snapbot

   #6

   23 guests... Right!
   
   17 Snapbots,
   1 exabot,
   1 Shim-Crawler,
   4 Internet Explorer users.
   
   That is 4 guests, not 23.

7. Re: Snapbot

   #7

   Snapbot uses the IP block 38.98.19.xxx
   
   Ban?

8. Re: Snapbot

   #8

   I'm all for it.

9. Re: Snapbot

   #9

   I will modify the script to count them as spiders. I am not entirely convinced from the evidence I have seen that they are malicious and don't follow robots.txt. I have found it to the contrary in many cases...and there is not much on the main site that is restricted to the bots. Forum has some nooks but they are member only anyway so a bot would get rejected.

10. Re: Snapbot

    #10

    Thumbs up for the script change!
    
    As for maliciousness and such, I agree. Just threw the links up so you needen't do a lot of research on your own. Removing them from the user list was all I wanted.

11. Re: Snapbot

    #11

    Updated this, let me know if you see it not working.

Administrators · All categories · Guildhall