search

fingerprintjs / BotD

Bot detection library that runs in the browser. Detects automation tools and frameworks. No server required, runs 100% on the client. MIT license, no usage restrictions.
https://fingerprintjs.github.io/BotD/main/
MIT License
954 stars 77 forks source link

Encountering consistent bot detection issue on UC browser (https://www.uc.cn/) for Android #130

Closed phongdemon closed 7 months ago

phongdemon commented 1 year ago

Hi, I've noticed that the function getBrowserEngineKind() always returns Unknown when using the UC browser on an Android device, specifically version 15.5.1. This browser is highly popular in China. Is there any way to address this issue?

xnerhu commented 1 year ago

Hello. We will investigate this and get back to you soon.

xnerhu commented 11 months ago

@phongdemon please, go to https://fingerprintjs.github.io/BotD/main/ and copy the logs here.

xnerhu commented 11 months ago

I've downloaded an apk from https://apkpure.com/pl/reviews/com.UCMobile.intl as I could not force Play Store to allow me to download it directly from Google Play Store. I have chromium browser engine kind.

tvvocold commented 10 months ago

@xnerhu here is my log in Android 10 with UC browser, seems that it's similar to #139 

Collection time:
57.90ms
Detection time:
63.00ms
Detection result:
{
    "bot": true,
    "botKind": "headless_chrome"
}
Detectors:
{
    "detectAppVersion": {
        "bot": false
    },
    "detectDocumentAttributes": {
        "bot": false
    },
    "detectErrorTrace": {
        "bot": false
    },
    "detectEvalLengthInconsistency": {
        "bot": true,
        "botKind": "unknown"
    },
    "detectFunctionBind": {
        "bot": false
    },
    "detectLanguagesLengthInconsistency": {
        "bot": false
    },
    "detectNotificationPermissions": {
        "bot": false
    },
    "detectPluginsArray": {
        "bot": false
    },
    "detectPluginsLengthInconsistency": {
        "bot": true,
        "botKind": "headless_chrome"
    },
    "detectProcess": {
        "bot": false
    },
    "detectUserAgent": {
        "bot": false
    },
    "detectWebDriver": {
        "bot": false
    },
    "detectWebGL": {
        "bot": false
    },
    "detectWindowExternal": {
        "bot": false
    },
    "detectWindowSize": {
        "bot": false
    },
    "detectMimeTypesConsistent": {
        "bot": false
    },
    "detectProductSub": {
        "bot": false
    },
    "detectDistinctiveProperties": {
        "bot": false
    }
}
Debug data:
{
    "browserEngineKind": "unknown",
    "browserKind": "chrome",
    "documentFocus": false,
    "mozAppearanceSupport": false,
    "isAndroid": false,
    "isDesktopSafari": false
}
Collected data:
{
    "process": {
        "state": -1,
        "error": "BotdError: window.process is undefined"
    },
    "userAgent": {
        "value": "Mozilla/5.0 (Linux; U; Android 10; zh-CN; PDEM30 Build/QKQ1.191222.002) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/100.0.4896.58 UCBrowser/15.5.8.1248 Mobile Safari/537.36",
        "state": 0
    },
    "appVersion": {
        "value": "5.0 (Linux; U; Android 10; zh-CN; PDEM30 Build/QKQ1.191222.002) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/100.0.4896.58 UCBrowser/15.5.8.1248 Mobile Safari/537.36",
        "state": 0
    },
    "rtt": {
        "value": 0,
        "state": 0
    },
    "windowSize": {
        "value": {
            "outerWidth": 412,
            "outerHeight": 773,
            "innerWidth": 412,
            "innerHeight": 773
        },
        "state": 0
    },
    "pluginsLength": {
        "value": 0,
        "state": 0
    },
    "pluginsArray": {
        "value": true,
        "state": 0
    },
    "errorTrace": {
        "value": "TypeError: Cannot read properties of null (reading '0')\n    at errorTrace (https://fingerprintjs.github.io/BotD/main/main.bundle.js:1:28066)\n    at n.<anonymous> (https://fingerprintjs.github.io/BotD/main/main.bundle.js:1:33600)\n    at https://fingerprintjs.github.io/BotD/main/main.bundle.js:1:21466\n    at Object.next (https://fingerprintjs.github.io/BotD/main/main.bundle.js:1:21571)\n    at https://fingerprintjs.github.io/BotD/main/main.bundle.js:1:20487\n    at new Promise (<anonymous>)\n    at e (https://fingerprintjs.github.io/BotD/main/main.bundle.js:1:20232)\n    at https://fingerprintjs.github.io/BotD/main/main.bundle.js:1:33437\n    at Array.map (<anonymous>)\n    at n.<anonymous> (https://fingerprintjs.github.io/BotD/main/main.bundle.js:1:33413)",
        "state": 0
    },
    "productSub": {
        "value": "20030107",
        "state": 0
    },
    "windowExternal": {
        "value": "[object External]",
        "state": 0
    },
    "mimeTypesConsistent": {
        "value": true,
        "state": 0
    },
    "evalLength": {
        "value": 33,
        "state": 0
    },
    "webGL": {
        "value": {
            "vendor": "WebKit",
            "renderer": "WebKit WebGL"
        },
        "state": 0
    },
    "webDriver": {
        "value": false,
        "state": 0
    },
    "languages": {
        "value": [
            [
                "zh-CN"
            ],
            [
                "zh-CN",
                "en-US"
            ]
        ],
        "state": 0
    },
    "notificationPermissions": {
        "state": -1,
        "error": "BotdError: window.Notification is undefined"
    },
    "documentElementKeys": {
        "value": [
            "lang"
        ],
        "state": 0
    },
    "functionBind": {
        "value": "function bind() { [native code] }",
        "state": 0
    },
    "distinctiveProps": {
        "value": {
            "awesomium": false,
            "cef": false,
            "cefsharp": false,
            "coachjs": false,
            "fminer": false,
            "geb": false,
            "nightmarejs": false,
            "phantomas": false,
            "phantomjs": false,
            "rhino": false,
            "selenium": false,
            "webdriverio": false,
            "webdriver": false,
            "headless_chrome": false
        },
        "state": 0
    }
}
xnerhu commented 10 months ago

Thank you @tvvocold . It all seems to be an issue with pluginsLengthInconsistency.

phongdemon commented 10 months ago

@xnerhu here is my log in Android 10 with UC browser, seems that it's similar to #139

Collection time:
57.90ms
Detection time:
63.00ms
Detection result:
{
    "bot": true,
    "botKind": "headless_chrome"
}
Detectors:
{
    "detectAppVersion": {
        "bot": false
    },
    "detectDocumentAttributes": {
        "bot": false
    },
    "detectErrorTrace": {
        "bot": false
    },
    "detectEvalLengthInconsistency": {
        "bot": true,
        "botKind": "unknown"
    },
    "detectFunctionBind": {
        "bot": false
    },
    "detectLanguagesLengthInconsistency": {
        "bot": false
    },
    "detectNotificationPermissions": {
        "bot": false
    },
    "detectPluginsArray": {
        "bot": false
    },
    "detectPluginsLengthInconsistency": {
        "bot": true,
        "botKind": "headless_chrome"
    },
    "detectProcess": {
        "bot": false
    },
    "detectUserAgent": {
        "bot": false
    },
    "detectWebDriver": {
        "bot": false
    },
    "detectWebGL": {
        "bot": false
    },
    "detectWindowExternal": {
        "bot": false
    },
    "detectWindowSize": {
        "bot": false
    },
    "detectMimeTypesConsistent": {
        "bot": false
    },
    "detectProductSub": {
        "bot": false
    },
    "detectDistinctiveProperties": {
        "bot": false
    }
}
Debug data:
{
    "browserEngineKind": "unknown",
    "browserKind": "chrome",
    "documentFocus": false,
    "mozAppearanceSupport": false,
    "isAndroid": false,
    "isDesktopSafari": false
}
Collected data:
{
    "process": {
        "state": -1,
        "error": "BotdError: window.process is undefined"
    },
    "userAgent": {
        "value": "Mozilla/5.0 (Linux; U; Android 10; zh-CN; PDEM30 Build/QKQ1.191222.002) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/100.0.4896.58 UCBrowser/15.5.8.1248 Mobile Safari/537.36",
        "state": 0
    },
    "appVersion": {
        "value": "5.0 (Linux; U; Android 10; zh-CN; PDEM30 Build/QKQ1.191222.002) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/100.0.4896.58 UCBrowser/15.5.8.1248 Mobile Safari/537.36",
        "state": 0
    },
    "rtt": {
        "value": 0,
        "state": 0
    },
    "windowSize": {
        "value": {
            "outerWidth": 412,
            "outerHeight": 773,
            "innerWidth": 412,
            "innerHeight": 773
        },
        "state": 0
    },
    "pluginsLength": {
        "value": 0,
        "state": 0
    },
    "pluginsArray": {
        "value": true,
        "state": 0
    },
    "errorTrace": {
        "value": "TypeError: Cannot read properties of null (reading '0')\n    at errorTrace (https://fingerprintjs.github.io/BotD/main/main.bundle.js:1:28066)\n    at n.<anonymous> (https://fingerprintjs.github.io/BotD/main/main.bundle.js:1:33600)\n    at https://fingerprintjs.github.io/BotD/main/main.bundle.js:1:21466\n    at Object.next (https://fingerprintjs.github.io/BotD/main/main.bundle.js:1:21571)\n    at https://fingerprintjs.github.io/BotD/main/main.bundle.js:1:20487\n    at new Promise (<anonymous>)\n    at e (https://fingerprintjs.github.io/BotD/main/main.bundle.js:1:20232)\n    at https://fingerprintjs.github.io/BotD/main/main.bundle.js:1:33437\n    at Array.map (<anonymous>)\n    at n.<anonymous> (https://fingerprintjs.github.io/BotD/main/main.bundle.js:1:33413)",
        "state": 0
    },
    "productSub": {
        "value": "20030107",
        "state": 0
    },
    "windowExternal": {
        "value": "[object External]",
        "state": 0
    },
    "mimeTypesConsistent": {
        "value": true,
        "state": 0
    },
    "evalLength": {
        "value": 33,
        "state": 0
    },
    "webGL": {
        "value": {
            "vendor": "WebKit",
            "renderer": "WebKit WebGL"
        },
        "state": 0
    },
    "webDriver": {
        "value": false,
        "state": 0
    },
    "languages": {
        "value": [
            [
                "zh-CN"
            ],
            [
                "zh-CN",
                "en-US"
            ]
        ],
        "state": 0
    },
    "notificationPermissions": {
        "state": -1,
        "error": "BotdError: window.Notification is undefined"
    },
    "documentElementKeys": {
        "value": [
            "lang"
        ],
        "state": 0
    },
    "functionBind": {
        "value": "function bind() { [native code] }",
        "state": 0
    },
    "distinctiveProps": {
        "value": {
            "awesomium": false,
            "cef": false,
            "cefsharp": false,
            "coachjs": false,
            "fminer": false,
            "geb": false,
            "nightmarejs": false,
            "phantomas": false,
            "phantomjs": false,
            "rhino": false,
            "selenium": false,
            "webdriverio": false,
            "webdriver": false,
            "headless_chrome": false
        },
        "state": 0
    }
}

@xnerhu: This is indeed my issue. You need to download UC version 15.5 from the official website (https://www.uc.cn/), not the version on the Play Store.

xnerhu commented 10 months ago

Thanks @phongdemon .

xnerhu commented 10 months ago

We're investigating if disabling navigator plugins detector would increase the number of false positives / reduce the number of true positives

tvvocold commented 10 months ago

We're investigating if disabling navigator plugins detector would increase the number of false positives / reduce the number of true positives

Any update?

tvvocold commented 10 months ago

Hi @xnerhu, It appears that the issue has not yet been resolved. Here is the log when I open the https://fingerprintjs.github.io/BotD/main/ Log: https://pastebin.com/raw/KFWFGUtp

xnerhu commented 10 months ago

Thank you. It appears that the issue lies in user agent parsing on our side, which thinks that it's a chrome browser instead of unknown one. Indeed, UA in this case has chrome in it and that's why it is considered chrome browser, which is of course incorrect. I've added additional check to detectPluginsLengthInconsistency that also checks browser engine kind and make sure it's chromium.

tvvocold commented 9 months ago

@xnerhu The result of the latest BotD v1.6.5 test is still a false positive, and this is the log: https://pastebin.com/3H3QDknG

xnerhu commented 7 months ago

@tvvocold can you confirm that #145 fixed it? It works on my android uc browser

tvvocold commented 7 months ago

@tvvocold can you confirm that #145 fixed it? It works on my android uc browser

Yeah, it works fine, bug fixed : )

xnerhu commented 7 months ago

Great! Thanks