Currently the site does not have a robots.txt file. Consequences of not having this file is that crawlers may be indexing pages that do not need or should not be indexed. In addition, it may prevent the website from being overloaded with requests.
The solution would be to create a robots.txt in the root of the website and adding rules. Some content that could be disallowed is media content and live project URLs.
Description
Currently the site does not have a robots.txt file. Consequences of not having this file is that crawlers may be indexing pages that do not need or should not be indexed. In addition, it may prevent the website from being overloaded with requests.
The solution would be to create a robots.txt in the root of the website and adding rules. Some content that could be disallowed is media content and live project URLs.
Steps to Reproduce
Expected Result
The validation tool is able to find the robots.txt file and there are no warnings or errors associated with the file.
Actual Result
There is a warning that states that the robots.txt file was not found.
Environments
This issue is present on all browsers across all environments.
Screenshots