search

govCMS / audit-site

govCMS site audit and remediation tool. Used to verify all sites built in, and for GovCMS SaaS meet the minimum requirements.
4 stars 4 forks source link

Robots.txt is not set correctly #6

Open fiasco opened 6 years ago

fiasco commented 6 years ago

The command line output for this one didn't give enough information to know what the actual problem was. Also it seems like it might be a false positive since in my case the robotstxt variable was actually already set and I was still getting this error.

rooby commented 6 years ago

Also worth noting that this check causes false positives if you have legitimately modified the robots.txt variable in the database.

E.g. I added some additional pages to the robots.txt and it tripped this check.

OPCIT commented 6 years ago

June 8 2018 and this is still being reported as an issue without any guidance on what needs to be done.

Audit Site reports: Robots.txt is not set correctly, currently set to #...... but then goes on to list the standard robots.txt file that is shipped with the distro.

Did I miss something?

tobybellwood commented 6 years ago

We're just about to start working on the 2.x version of this - we'll make sure that this check is fixed up there!