Data needed!

[MShawon]

Is there an existing issue for this?

• [X] I have searched the existing issues

I'm submitting a ...

• [ ] bug report
• [ ] feature request
• [ ] support request --> Contact me over mail for support https://github.com/MShawon

Description

I need several Windows, Linux, macOS sec-ch-ua information to randomize user agents more accurately You can go to this Link and take a screenshot like this and post here or send it to my email

Also, go there Link to take a ss like this

Environment

- OS : 
- Python : 
- Script version :

config.json

N/A

[mercuryin]

[Percy1082]

[ccgr]

[sebbourgeois]

All are Desktop versions

Chrome :

Firefox :

Edge :

[kattstof]

Here's a list of almost 4k useragents https://pastebin.com/Gvscw7kc

[JijaProGamer]

Hey @MShawon , I found pretty interesting stuff while rewriting my view bot, you should check it out.

It turns out, that my theory that google makes a big list of your data for a fingerprint. I was reading around requests made by firefox, and I found this interesting bit when starting the browser

{
  "context": {
    "client": {
      "hl": "en",
      "gl": "RO",
      "remoteHost": "<redacted IP>",
      "deviceMake": "",
      "deviceModel": "",
      "visitorData": "CgtRU2gzTDRCVU50cyj40JaoBjIGCgJSTxIA",
      "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/115.0,gzip(gfe)",
      "clientName": "WEB",
      "clientVersion": "2.20230914.04.00",
      "osName": "Windows",
      "osVersion": "10.0",
      "originalUrl": "https://www.youtube.com/watch?v=7UPEJjZ0UAE&pp=YAHIAQE%3D",
      "platform": "DESKTOP",
      "clientFormFactor": "UNKNOWN_FORM_FACTOR",
      "configInfo": {
        "appInstallData": "CPjQlqgGEKfq_hIQ1uqvBRC4i64FEIbqrwUQiOOvBRDk5q8FENPhrwUQrtT-EhDrk64FENShrwUQ-uT-EhDqw68FEJTo_hIQl-f-EhCj3q8FEInorgUQzK7-EhC1pq8FENXlrwUQ3euvBRC15q8FEPq-rwUQxN2vBRDnuq8FEKy3rwUQx-avBRC7668FEJrwrwUQ3ej-EhDM364FELzM_hIQ6-j-EhC24K4FEOLUrgUQ5LP-EhCMy68FEKXC_hIQ26-vBRDuoq8FELfq_hIQ2cmvBRDyqK8FEKbs_hIQj_WvBRC9tq4FELTJrwUQ0OWvBRDR7P4SEKDerwU%3D"
      },
      "browserName": "Firefox",
      "browserVersion": "115.0",
      "acceptHeader": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8",
      "deviceExperimentId": "ChxOekkzT1RRd09UWXdOakU1TVRVMU5ETTROdz09EPjQlqgGGPjQlqgG",
      "screenWidthPoints": 1280,
      "screenHeightPoints": 720,
      "screenPixelDensity": 1,
      "screenDensityFloat": 1,
      "utcOffsetMinutes": 180,
      "userInterfaceTheme": "USER_INTERFACE_THEME_LIGHT",
      "clientScreen": "WATCH",
      "mainAppWebInfo": {
        "graftUrl": "/watch?v=7UPEJjZ0UAE&pp=YAHIAQE%3D",
        "pwaInstallabilityStatus": "PWA_INSTALLABILITY_STATUS_UNKNOWN",
        "webDisplayMode": "WEB_DISPLAY_MODE_BROWSER",
        "isWebNativeShareAvailable": false
      },
      "timeZone": "Europe/Bucharest"
    },
    "user": {
      "lockedSafetyMode": false
    },
    "request": {
      "useSsl": true,
      "internalExperimentFlags": [
        {
          "key": "force_enter_once_in_webview",
          "value": "true"
        }
      ],
      "consistencyTokenJars": []
    },
    "clickTracking": {
      "clickTrackingParams": "CKUBENwwIhMIz7GQr5mvgQMVZd0RCB2IZATjMgpnLWhpZ2gtcmVjWg9GRXdoYXRfdG9fd2F0Y2iaAQYQjh4YngE="
    },
    "adSignalsInfo": {
      "params": [
        {
          "key": "dt",
          "value": "1694869625531"
        },
        {
          "key": "flash",
          "value": "0"
        },
        {
          "key": "frm",
          "value": "0"
        },
        {
          "key": "u_tz",
          "value": "180"
        },
        {
          "key": "u_his",
          "value": "1"
        },
        {
          "key": "u_h",
          "value": "720"
        },
        {
          "key": "u_w",
          "value": "1280"
        },
        {
          "key": "u_ah",
          "value": "720"
        },
        {
          "key": "u_aw",
          "value": "1280"
        },
        {
          "key": "u_cd",
          "value": "24"
        },
        {
          "key": "bc",
          "value": "31"
        },
        {
          "key": "bih",
          "value": "720"
        },
        {
          "key": "biw",
          "value": "1280"
        },
        {
          "key": "brdim",
          "value": "4,4,4,4,1280,0,1280,720,1280,720"
        },
        {
          "key": "vis",
          "value": "1"
        },
        {
          "key": "wgl",
          "value": "true"
        },
        {
          "key": "ca_type",
          "value": "image"
        }
      ]
    },
    "videoId": "7UPEJjZ0UAE",
    "params": "YAHIAQE%3D",
    "playbackContext": {
      "contentPlaybackContext": {
        "currentUrl": "/watch?v=7UPEJjZ0UAE&pp=YAHIAQE%3D",
        "vis": 5,
        "splay": false,
        "autoCaptionsDefaultOn": false,
        "autonavState": "STATE_NONE",
        "html5Preference": "HTML5_PREF_WANTS",
        "signatureTimestamp": 19612,
        "autoplay": true,
        "autonav": true,
        "referer": "https://www.youtube.com/",
        "lactMilliseconds": "-1",
        "watchAmbientModeContext": {
          "watchAmbientModeEnabled": true
        }
      }
    },
    "racyCheckOk": false,
    "contentCheckOk": false
  }
}

[MShawon]

@kattstof @JijaProGamer how do you set navigator.webdriver to false in selenium geckodriver? @kattstof since you mentioned Firefox works for you. Did you use the same fingerprinting extension I used? Or something different?

Sec-cha-ua is not supported by Firefox but all chromium browsers support it. So modifying user agent only is a dead end for chromium based browsers. Although I disabled useragentclienthint to avoid sec-ch-ua data, it's not working.

[JijaProGamer]

@MShawon

Firstly you add some code that runs on new documents, and the code should set navigator.webdriver to false:

code = "Object.defineProperty(navigator, 'webdriver', { value: false, configurable: false, enumerable: false, writable: false })"

driver.execute_cdp_cmd('Page.addScriptToEvaluateOnNewDocument', {'source': code})

EDIT: thats for chromium only, for firefox you can also use

driver.executeScript(code)

[MShawon]

@JijaProGamer But selenium Firefox doesn't support driver.execute_cdp_cmd()

[JijaProGamer]

@JijaProGamer But selenium Firefox doesn't support driver.execute_cdp_cmd()

Yeah, Ive sent a fix rn

[kattstof]

i just used basic mitigation techniques with Firefox. If you need to use chrome however, undetected_chromedriver has been updated within the last 3 weeks btw so you might try the new update, the version your using 3.2.1 is deprecated and no longer works and is detected by cloud flare and others.