search

omrilotan / isbot

🤖/👨‍🦰 Detect bots/crawlers/spiders using the user agent string
https://isbot.js.org/
The Unlicense
933 stars 77 forks source link

Bytespider was not recognized #268

Closed Woorooo closed 2 months ago

Woorooo commented 2 months ago

User Agent String

Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; spider-feedback@bytedance.com)

Reproduce

When integrated via https://www.jsdelivr.com/package/npm/isbot CDN, through both UMD and ESM. The real Bytespider is never recognized. However when faking the user string via Chrome extension, it does recognize it.

omrilotan commented 2 months ago

If you're using ibot on the browser, it makes sense. Because browser using older versions of Javascript use a more simplistic pattern. See https://github.com/omrilotan/isbot?tab=readme-ov-file#fallback However, I will be inclined to add the substring "spider" to the simple pattern and check how it affects its efficiency. I will try to look into it tonight

Woorooo commented 2 months ago

It looks like substring "spider" is already in the simple pattern detection. Can the issue be caused by something else?

On Fri, Aug 9, 2024, 12:27 omrilotan @.***> wrote:

If you're using ibot on the browser, it makes sense. Because browser using older versions of Javascript use a more simplistic pattern. See https://github.com/omrilotan/isbot?tab=readme-ov-file#fallback However, I will be inclined to add the substring "spider" to the simple pattern and check how it affects its efficiency. I will try to look into it tonight

— Reply to this email directly, view it on GitHub https://github.com/omrilotan/isbot/issues/268#issuecomment-2277641854, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALOXDI3NLDYR5GMBJANYOXLZQSKSJAVCNFSM6AAAAABMIE5NDOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENZXGY2DCOBVGQ . You are receiving this because you authored the thread.Message ID: @.***>

omrilotan commented 2 months ago

Maybe it doesn't support the Navigator interface. Would you like to share steps to reproduce?

Woorooo commented 2 months ago

Sorry, I guess I made a mistake. It seems that isbot script is not being loaded by Bytespider at all, so there is no response. I forgot that I was logging only detected bots, so those that block js files will not be identified.