search

omrilotan / isbot

🤖/👨‍🦰 Detect bots/crawlers/spiders using the user agent string
https://isbot.js.org/
The Unlicense
905 stars 74 forks source link

Google Pixel phone was wrongly recognised as a bot #190

Closed jgabriele closed 2 years ago

jgabriele commented 2 years ago

User Agent String

Safari/600.2.5 MobileRicardo (Google Pixel 6; android 13) ricardo.ch/8.28.2-82802 (debug) deviceId/dfa51331-fdeb-46d7-aa01-8d09a263ef90

Reproduce

https://isbot.js.org/?ua=Safari/600.2.5%20MobileRicardo%20(Google%20Pixel%206;%20android%2013)%20ricardo.ch/8.28.2-82802%20(debug)%20deviceId/dfa51331-fdeb-46d7-aa01-8d09a263ef90

omrilotan commented 2 years ago

Thank you for this issue. I can address it promptly.

omrilotan commented 2 years ago

@jgabriele would you mind specifying what device and browser this is?

Chrome's user agent looks different than this, and the device name does not include the "Google" substring.

Mozilla/5.0 (Linux; Android 13; Pixel 6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Mobile Safari/537.36
jgabriele commented 2 years ago

Hello, indeed it's not Chrome's user agent, there it is fine. It's the Webview on Android for those Pixel phones that seems to send this UA string.

I do not know how to prepare a minimal reproducible case, we found that issue on our native app

Please let me know if I can help somehow

jgabriele commented 2 years ago

EDIT @omrilotan : I might have open an issue too fast, I do not know if this UA is coming from the Webview or if our Android mates are building it themselves. Please don't waste some time for now I will ping you when it's clarified.

BTW, impressive reaction time, it's very nice to see so active maintainers, you rock!

omrilotan commented 2 years ago

I've bundled this issue with another change I've had lined up. I also found a user agent that matches the same issue (older Pixel version, but still):

Mozilla/5.0 (Linux; Android 7.1.2; google Pixel 2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Mobile Safari/537.36

Feel free to take a look at the changeset. I can release it today as soon as my tests pass https://github.com/omrilotan/isbot/pull/189

jgabriele commented 2 years ago

So actually we looked into the problem some more, with the help of our mobile developers. It seems this is not coming directly from Android OS, but rather it's a custom User Agent we construct ourselves in the mobile apps, then inject into the WebView.

The user agents is composed of the Device's Manufacturer and Model name, resulting in: Google Pixel 6 Asus Zenfon 9 etc.

In our case the issue lies on our side, but out of curiosity how to reproduce your "google Pixel 2" UA? On which device is it sent?

omrilotan commented 2 years ago

I have access to appr a billion traffic logs each day, so I was able to find some traffic with this old Google Pixel 2 user agent string. It is a bit of an edge case, but it sounds to me like an issue that can easily happen to others. Since it originates in honest user-agent building protocols. I think it's good to resolve it.

Resolved: https://isbot.js.org/?ua=Safari/600.2.5%20MobileRicardo%20(Google%20Pixel%206;%20android%2013)%20ricardo.ch/8.28.2-82802%20(debug)%20deviceId/dfa51331-fdeb-46d7-aa01-8d09a263ef90

jgabriele commented 2 years ago

We've been trying your new release on our app, it works now 🎉.

Fastest. Issue fix. Ever. Thank you ❤️ !