search

FreeUKGen / FreeUKGenealogy

Apache License 2.0
0 stars 0 forks source link

robots.txt - Disallow: /search #52

Open ghost opened 5 years ago

ghost commented 5 years ago

@DeniseColbert Any reason we're not blocking search engine indexing for /search URL's? (Lemon noticed bingbot traffic - unless there is a known reason for allowing /search URL's to be indexed, best practices would discourage search engine spiders from this portion of the site)

DeniseColbert commented 5 years ago

No reason I'm aware of... @PatReynolds do you know of any?

DeniseColbert commented 5 years ago

This has been raised again: We're having an (ongoing) issue with users landing on https://www.freeukgenealogy.org.uk/search and trying to search for their ancestors using the search bar, rather than going to the projects via the buttons. We think they're coming from: image

Gray's options:

  1. Remove /search and all pages under /search from Google's index (removes all traces of /search from Google results)
  2. Create a separate page to direct to ancestry searches and tell Google that /search should actually refer to that URL (index gets updated to /ancestry-search or whatever the new URL is)
  3. Mix of 1 & 2 - leave /search in Google's index (pointing to the new URL) and remove pages under /search from the index Gray's preference

However we don't (/can't?) have one page where you can search your ancestors; our records are on three separate websites.