Closed Aariq closed 7 months ago
This adds a robots.txt that I've been using for my personal site. Source: https://neil-clarke.com/block-the-bots-that-feed-ai-models-by-scraping-your-website/
I didn't know this was possible, this is great!
This delights me.
This adds a robots.txt that I've been using for my personal site. Source: https://neil-clarke.com/block-the-bots-that-feed-ai-models-by-scraping-your-website/