Prevent general public and search engines from accessing site

[bkiahstroud]

Acceptance Criteria

• [ ] Only certain users are allowed to access the site
• [ ] Search engines are blocked from indexing the site

Implementation Suggestions

Use nginx to implement HTTP basic auth (docs)

Use robots.txt and possibly HTML meta tags

[bkiahstroud]

I deployed this to production, but I don't think it's working. I say this because /robots.txt throws a 404 error.

The page may just need to be "hooked in" so that the server serves the file, but that's a guess