Add robots.txt file to disallow indexing of /entries/*

StephenHaney / dear-diary-ai

Turn your journal into music, winner in the Magenta/Gray Area ML Music Hackathon

https://deardiary.ai

MIT License

99 stars 13 forks source link

Add robots.txt file to disallow indexing of /entries/* #21

Open qurbat opened 2 years ago

qurbat commented 2 years ago

I know that entries are meant to be public if the URL is shared with someone, however, currently, the search query for site:deardiary.ai inurl:entry on Google indexes at least 5 entries, which in my opinion should not be visible, even if the entries have been linked on a public website. Creating a robots.txt file which disallows search engine indexing of /entries/* should take care of this. Please let me know if further information is required. Thank you!

StephenHaney commented 2 years ago

Great idea, thanks for bringing this up.