add a setting to control whether pages are indexed by Google or now

mysociety / yournextrepresentative

A website for crowd-sourcing structured election candidate data

https://candidates.democracyclub.org.uk/

GNU Affero General Public License v3.0

56 stars 21 forks source link

Open mhl opened 7 years ago

mhl commented 7 years ago

This involves adding a robots.txt file and adding a <meta name="robots" ...> tag into each page: https://developers.google.com/webmasters/control-crawl-index/docs/robots_meta_tag

This new setting should be turned on candidates.staging.mysociety.org, for example.