dtekcth / dfotose

DFotos hemsida

BSD 3-Clause "New" or "Revised" License

0 stars 3 forks source link

Add a robots.txt to make dfoto.se a bit harder to find #19

Closed Rembane closed 2 years ago

Rembane commented 6 years ago

Some sources:

https://support.google.com/webmasters/answer/6062608?hl=en https://en.wikipedia.org/wiki/Robots_exclusion_standard#About_the_standard

Example of robots.txt that probably does what we want

User-agent: *
Disallow: /

BionicRiddle commented 2 years ago

Fixed in https://github.com/dtekcth/dfotose/commit/c705bf5d27728d49a6c5031df5fbdfa0413c1e85 Closing Issue