[Feat]: Extend hideFromBots function

orestbida / cookieconsent

:cookie: Simple cross-browser cookie-consent plugin written in vanilla js

https://playground.cookieconsent.orestbida.com/

MIT License

3.68k stars 387 forks source link

[Feat]: Extend hideFromBots function #687

Open HemiCZ opened 1 month ago

HemiCZ commented 1 month ago

Description

The current function only checks bots with user-agent like /bot|crawl|spider|slurp|teoma/i.

However, Google uses other bots with different names, such as:

Google-InspectionTool
GoogleOther
Google-Extended
Chrome-Lighthouse

https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers#googlebot-desktop https://stackoverflow.com/a/78137919

Proposed solution

No response

Additional details

No response

orestbida commented 4 weeks ago

All the user agents strings for the services you listed contain the "bot" key, which is already included in the regex.

HemiCZ commented 4 weeks ago

@orestbida I'm not sure if I understand it, but in "Google-InspectionTool" is no "bot" key, or?

orestbida commented 3 weeks ago

My bad, I didn't check the second half of the services (the ones you actually listed). You are right, the plugin won't detect these bots.

With that said I think it's fine keeping it as is since the main point of this option is to prevent the indexing of the modal's content (indexing is done by other bots afaik).