Agent with '+http...' almost always a bot.

JayBizzle / Crawler-Detect

🕷 CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent

https://crawlerdetect.io

MIT License

1.93k stars 254 forks source link

Agent with '+http...' almost always a bot. #501

Open bartonlp opened 11 months ago

bartonlp commented 11 months ago

Any time you find a user agent with '+http' it is almost always a bot. So for example, '+https://help.something.com' will probably be a bot. The 'good' bots almost always have a place where you can see who they are.

JayBizzle commented 11 months ago

"Probably" is a bit loose for my liking, unless you can convince me otherwise 🤔