Robots 2.6

Description:
Disallows robots from indexing your site unless explicitly allowed.

(Robots link in Visitors tab)
[Image unavailable; source: https://xenforo.com/community/attachments/pic001-jpg.157056/]


(Example of Robots page)
[Image unavailable; source: https://xenforo.com/community/attachments/pic002-jpg.176942/]


(Example of Options page)
[Image unavailable; source: https://xenforo.com/community/attachments/pic003-jpg.178451/]


(Example of User group permissions)
[Image unavailable; source: https://xenforo.com/community/attachments/pic004-jpg.157059/]


Features:

• All phrases start with robots_ for your convenience.

User group permissions:
Normally only the Administrative user group is set the 'Yes' in order to see the Robots link in the visitor tab menu.

Questions and Answers:
Q: What is the purpose of this add-on?
A: Lots of robots crawling your forum will slow it down.
Q: I noticed a robot crawling my site but it's identified as a guest.
A: You need to add the name to the Blacklist field in the Options page.
Q: What happens if a robot which is not allowed comes to my site?
A: It receives a blank page.
Q: Why not use the robots.txt file to restrict robots?
A: Because robots ignore this file.

Database modifications:
This add-on will create a table called xf_robots in your database.