search

CodeWithAloha / uipa

Helping submit, track, and share public records requests in Hawaii
http://uipa.org
MIT License
10 stars 6 forks source link

beep boop: robots.txt #87

Open tyliec opened 8 months ago

tyliec commented 8 months ago

Adds a robots.txt file and it's related URL route.

Explanation: https://www.cloudflare.com/learning/bots/what-is-robots-txt/

#  __    __   __  .______      ___      
# |  |  |  | |  | |   _  \    /   \     
# |  |  |  | |  | |  |_)  |  /  ^  \    
# |  |  |  | |  | |   ___/  /  /_\  \   
# |  `--'  | |  | |  |     /  _____  \  
#  \______/  |__| | _|    /__/     \__\