search

djstevenson / songs-to-the-siren

A blog about songs
MIT License
0 stars 0 forks source link

Fix indexing by Google etc of all tag combinations via robots.txt #349

Closed djstevenson closed 3 years ago

djstevenson commented 3 years ago

Bots are indexing every combination of tags, as tag buttons are "GET" operations to add a tag to a search.

Either prevent these searches using robots.txt, or make them a POST. Or both.

djstevenson commented 3 years ago

NOTE: It probably should not be a POST as it alters no state. If we can fix this indexing problem with robots.txt, then that's the right way to go.

djstevenson commented 3 years ago

Optional alternative:

Do not accumulate tags. When you click on a tag, that's what you see. This probably makes most sense for a small site like this.