search

JayBizzle / Crawler-Detect

🕷 CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent
https://crawlerdetect.io
MIT License
1.94k stars 254 forks source link

Missing bot detection for prerender.io's crawler #440

Closed LinusGeffarth closed 3 years ago

LinusGeffarth commented 3 years ago

I'm more confident this time b/c the web demo doesn't detect the following as a bot, even though it is:

Prerender

... from prerender.io. The web demo does detect the full user-agent because of "GitHub" and probably "HeadlessChrome":

Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/71.0.3578.98 Safari/537.36 Prerender (+https://github.com/prerender/prerender)

... but I figured it might be useful to also check for "Prerender" itself, in case the rest of the user-agent changes over time.

JayBizzle commented 3 years ago

Would you like to create a PR to add this bot!?

Thanks 👍

JayBizzle commented 3 years ago

Fixed in #441