Apocalypse
27th March 2004, 09:54 AM
does anyone have a htaccess file that will stop bots and spiders from going through and ripping the site of bandwidth? Google is fine, but some of the bots seem rather evil
|
View Full Version : htaccess file to stop bots and spiders Apocalypse 27th March 2004, 09:54 AM does anyone have a htaccess file that will stop bots and spiders from going through and ripping the site of bandwidth? Google is fine, but some of the bots seem rather evil KuraFire 27th March 2004, 01:53 PM Check the Inktomi Spider thread on vb.com for one. Apocalypse 27th March 2004, 04:51 PM thanks kura :) KuraFire 27th March 2004, 07:17 PM You're welcome. I'd have dug it up for you myself, but I have it on my home computer, and I'm at work atm... and going through a big thread for you, well, I figured you could do that yourself, don't need me to do ALL the work for you ;) Hope you find it! :) Apocalypse 28th March 2004, 01:25 AM not a problem. I found a few(50) or so threads, some were good some were not so good. I keep forgetting vb.com is more than a support site for bugs and install problems. I basicly go there to read the announcements and bug tracker(forum) not much else. |