Managment of ai.txt

[philipzaengle]

We're seeing more and more clients request the addition and management of an ai.txt file. While this isn't directly related to SEO it does fall into the indexing/metadata category and could be a helpful addition to SEOMatic. The current standard follows the same format as robots.txt, just in a different file (ai.txt).

The robots.txt standard allows for a good deal of flexibility in terms of saying what's allowed, perhaps something as simple as a default disallow all (as that's what many folks will want) or the ability to paste in the content from a service such as Spawning AI.txt

The alternative to this is for us to either use globals and allow customers to manage settings there or hard code the file itself.

Here's an image of a puppy using craft according to chat gpt.

[khalwat]

So I'll likely implement this, even though I think it's largely placebo, in that ai.txt isn't a widely adopted standard (or one originating from a standards body), and everything you can do in ai.txt you can already do via robots.txt

As always, any bots that abide by the rules can be cordoned off, and any that don't... well, you're out of luck regardless of where/how you try to block them.

I think integrating DarkVistors with the robots.txt implementation in SEOmatic is likely a more effective, practical solution: https://darkvisitors.com/

Links:

https://spawning.substack.com/p/aitxt-a-new-way-for-websites-to-set

https://github.com/menro/ai.txt

https://community.openai.com/t/proposal-transparent-ai-interaction-through-ai-txt-integration/322564

[philipzaengle]

Placebos make people feel good. 5/10 people agree.