khalwat
commented
6 months ago So I'll likely implement this, even though I think it's largely placebo, in that ai.txt isn't a widely adopted standard (or one originating from a standards body), and everything you can do in ai.txt you can already do via robots.txt
As always, any bots that abide by the rules can be cordoned off, and any that don't... well, you're out of luck regardless of where/how you try to block them.
I think integrating DarkVistors with the robots.txt implementation in SEOmatic is likely a more effective, practical solution: https://darkvisitors.com/
Links:
https://spawning.substack.com/p/aitxt-a-new-way-for-websites-to-set
https://github.com/menro/ai.txt
https://community.openai.com/t/proposal-transparent-ai-interaction-through-ai-txt-integration/322564
philipzaengle
We're seeing more and more clients request the addition and management of an
ai.txtfile. While this isn't directly related to SEO it does fall into the indexing/metadata category and could be a helpful addition to SEOMatic. The current standard follows the same format asrobots.txt,just in a different file (ai.txt).The robots.txt standard allows for a good deal of flexibility in terms of saying what's allowed, perhaps something as simple as a default disallow all (as that's what many folks will want) or the ability to paste in the content from a service such as Spawning AI.txt
The alternative to this is for us to either use globals and allow customers to manage settings there or hard code the file itself.
Here's an image of a puppy using craft according to chat gpt.