search

alaz / legitbot

🤔 Is this Web request from a real search engine🕷 or from an impersonating agent 🕵️‍♀️?
Other
22 stars 9 forks source link

Split Google crawlers into categories #107

Open alaz opened 11 months ago

alaz commented 11 months ago

https://developers.google.com/search/docs/crawling-indexing/verifying-googlebot