samwilson / twyne

A web-based journal for writing and photography.
GNU General Public License v3.0
8 stars 1 forks source link

Add a settings for robots.txt #178

Open revi opened 2 years ago

revi commented 2 years ago

I'd like to request crawlers to only look at P*** (that is, yes, posts only). (Maybe I want some posts be taken off crawlers' eye but not for now) Currently I cannot control that without a help of a robots.txt (or a meta tag in page header)

Note: I haven't tested uploading some robots.txt at public directory to see it just works as it is now, though.