search

solgenomics / sgn

The code behind the Sol Genomics Network, Cassavabase and other Breedbase websites
https://solgenomics.net
MIT License
66 stars 35 forks source link

Meta tags for webcrawlers #1466

Open dauglyon opened 6 years ago

dauglyon commented 6 years ago

Expected Behavior

We should have appropriate ROBOTS meta tags on pages which will help cut down bot traffic.

For Bugs:

Enviroment

Steps to Reproduce

lukasmueller commented 2 years ago

there is a robots.txt file but not sure about the meta tag. Seems to be an important addition.