Discourage search engines from indexing dev & staging directories

[richard015ar]

When Pressbooks Directory is searched in Google, the top result is the Staging environment version.

• [x] Add robots.txt nofollow directive to dev and staging directories to discourage search enginges from indexing.
• [x] Add unique metadata for the production directory to help search engines index the site (title, description, canonical link, keywords)
• [x] Submit main directory to https://www.google.com/webmasters/tools/submit-url
• [x] List up to 3 recommendations for SEO optimization for the Directory site

Psuedocode: If env !=prod echo <meta name="robots" content="noindex, nofollow" />

Ricardo to review. Oscar & Ricardo to test.

[richard015ar]

Recommendations for SEO optimizations:

• Improve the page speed. Here's the analysis result from PageSpeed: https://developers.google.com/speed/pagespeed/insights/?hl=es&url=https%3A%2F%2Fpressbooks.directory . There are a few things we could improve easly and others probably require more time.
• Create more links from other pages to Pressbooks Directory: we could add in the book's main page a link with a text like: "You can find this books in Pressbooks Directory" or "This book is part of the Pressbooks Directory", or something like: "Find similar books to this in the Pressbooks Directory", etc.
• Integrate social medias, i.e: add a button in the book card like "Share this book in Twitter/FB", for the whole site: "Share Pressbooks Directory in TW". There is a task for share search links.
• Use keywords in book images and other components.
• Use Google Search console

The production website was submitted to: https://search.google.com/search-console?resource_id=sc-domain%3Apressbooks.directory

[richard015ar]

It was deployed to Staging, it is ready for testing.

How to test

These metadata: https://github.com/pressbooks/pressbooks-book-directory-fe/pull/162/files#diff-7a7a37b12ee1265d7e27577ec286120d2cbc0940908635e264a2be44ccb9a8a0R72-R90 Should be present in the DOM (head section). And this metadata: https://github.com/pressbooks/pressbooks-book-directory-fe/pull/162/files#diff-7a7a37b12ee1265d7e27577ec286120d2cbc0940908635e264a2be44ccb9a8a0R101-R104 Should be present in Staging. Also, the robots.txt file should be present in https://staging.pressbooks.directory/robots.txt and https://dev.pressbooks.directory/robots.txt.

Once it is deployed to production:

• robots.txt should not be present and / or accessible
• This metadata: https://github.com/pressbooks/pressbooks-book-directory-fe/pull/162/files#diff-7a7a37b12ee1265d7e27577ec286120d2cbc0940908635e264a2be44ccb9a8a0R101-R104 should not be present in the DOM.