search

HalosGhost / halosgho.st

The repo backing my homepage
GNU General Public License v3.0
3 stars 0 forks source link

robots #18

Closed HalosGhost closed 5 years ago

HalosGhost commented 5 years ago

I do want my page to be indexed, but probably not universally.

Providing a reasonable robots.txt asset shouldn't be terribly difficult to do.

HalosGhost commented 5 years ago

Actually, I have come to the conclusion that this is unnecessary: bad bots will ignore it, and well-behaved bots don't really bother me having the site indexed.