>Please! Are you taking a special pleasure in making me ask all these >questions one after the other? :-) How can I do that? Furthermore, is >there a way to filter certain robots from certain pages?
Bekir, I can only answer what you ask, I cannot read your mind (grin). Please check out this link:
This link will tell you everything about the robot exclusion protocol. To keep ALL robots out, put this in your "robots .txt" file:
User-agent: * Disallow: /
You can limit access to certain robots too, by ALLOWing them by name. You can also allow access to portions of the site while DISALLOWING others.
>>>How can I use the exclude tag? >> >>Put this in between the <HEAD> and </HEAD> tags in your pages, and in >>your templates for dir.spml and topic.spml: >> >><meta name="ROBOTS" CONTENT="NOFOLLOW">
Change this to the following to keep robots from INDEXING rather than just FOLLOWING LINKS:
<meta name="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
>That, I already use, especially with pages carrying spml tags, but it >does not help to keep certain robots out.
--- Daniel O'Leary, Admin/WebMaster KloneZone Mac - A TeleFinder 5.7 Mac/Windows BBS
