This past week we have experienced lots of traffic comming from some bots that are reported to be taking down websites because of their aggressive behaviour.
This PR changes the robots.txt in goteo to disallow the robots of OpenAI (even though there are proves that it does not follow it), and Amazonbot, as well as other marketing tools.
:tophat: What? Why?
This past week we have experienced lots of traffic comming from some bots that are reported to be taking down websites because of their aggressive behaviour.
This PR changes the robots.txt in goteo to disallow the robots of OpenAI (even though there are proves that it does not follow it), and Amazonbot, as well as other marketing tools.
:hearts: Thank you!