search

alaz / legitbot

🤔 Is this Web request from a real search engine🕷 or from an impersonating agent 🕵️‍♀️?
Other
22 stars 9 forks source link

Split OpenAI bots #145

Closed alaz closed 1 week ago

alaz commented 1 month ago

They have published more extensive documentation on their crawlers: https://platform.openai.com/docs/bots

alaz commented 1 month ago

Still blocked on #131