This adds a small robots.txt only hiding the admin login form, which isn't publicly linked anyway, and result pages (in case someone posts their results URL somewhere on the web).
Most of the regular content is heavily cached to a degree that it could even be considered static. The only action triggering computations (POSTed to /jobs) is reCAPTCHA-protected anyway.
This adds a small robots.txt only hiding the admin login form, which isn't publicly linked anyway, and result pages (in case someone posts their results URL somewhere on the web). Most of the regular content is heavily cached to a degree that it could even be considered static. The only action triggering computations (POSTed to /jobs) is reCAPTCHA-protected anyway.