search

alaz / legitbot

🤔 Is this Web request from a real search engine🕷 or from an impersonating agent 🕵️‍♀️?
Other
24 stars 11 forks source link

Split OpenAI bots #145

Closed alaz closed 3 months ago

alaz commented 3 months ago

They have published more extensive documentation on their crawlers: https://platform.openai.com/docs/bots

alaz commented 3 months ago

Still blocked on #131