kaliiiiiiiiii / Selenium-Profiles

undetected Selenium using chromedriver and emulation / device profiles
Other
254 stars 28 forks source link

emulation leak in service-worker and new-tab #58

Open mehedihasanziku opened 1 year ago

mehedihasanziku commented 1 year ago

Describe the bug Navigator.hardwareConcurrency: does not match worker scope

To Reproduce

same as documentation

Expected behavior or error-message

ram and core should be probably match, unfortunately it fails to match or spoof both

Environment (please complete the following information):

Additional context Add any other context about the problem here.

image

kaliiiiiiiiii commented 1 year ago
  1. RAM and cores don't need to be the same, as RAM is the amount of GB available on your device (4), and cores the amount of cores your processor has (8)
  2. "does not match worker scope" is a bug inside chrome (see crbug#1358491) . Also, fixing this is allready in the TODO's
  3. Can you show me the error, when you click on the link at "lies(1)" ? image
mehedihasanziku commented 1 year ago
  1. Can you show me the error, when you click on the link at "lies(1)" ? image

@kaliiiiiiiiii It shows that error if RAM and cores mismatch, but technically both can be different.

kaliiiiiiiiii commented 1 year ago
  1. Can you show me the error, when you click on the link at "lies(1)" ? image

@kaliiiiiiiiii It shows that error if RAM and cores mismatch, but technically both can be different.

Oh now I think I got what you mean.

mehedihasanziku commented 1 year ago
  • Like for example if you emulate a different number of cores than your device actually has?

Yes, I have now cores-4 ram-4 After Selenium-Profiles cores-8 ram-4

  • Why do you think it has something to do with the RAM? With javascript, you'd get the value with navigator.deviceMemory, but there's not option to emulate that yet (crbug#1308873

It's just an thought, if i remove cores, no lies found Yes, no way, But I searched that, not sure how to implement

# https://stackoverflow.com/questions/59367665/spoof-navigator-properties-in-chrome-selenium-webdriver
driver.execute_cdp_cmd("Page.addScriptToEvaluateOnNewDocument", {
  "source": """
    Object.defineProperty(navigator, 'deviceMemory', {
      get: () => 99
    })
  """
})
kaliiiiiiiiii commented 1 year ago
# https://stackoverflow.com/questions/59367665/spoof-navigator-properties-in-chrome-selenium-webdriver
driver.execute_cdp_cmd("Page.addScriptToEvaluateOnNewDocument", {
  "source": """
    Object.defineProperty(navigator, 'deviceMemory', {
      get: () => 99
    })
  """
})

Yes, this is indeed an option, but creep-js is gonna detect it. Currently, I don't have a way to overwrite javascript values without the possibility of detection.

https://github.com/abrahamjuliot/creepjs/issues/147 or adtechmadness.wordpress.com/2019/03/23/javascript-tampering-detection-and-stealth might give you a starting point why overwriting isn't that easy

kaliiiiiiiiii commented 1 year ago

It's just an thought, if i remove cores, no lies found

yes, indeed. The emulation causes some other leaks as well inside worker scopes:

kaliiiiiiiiii commented 1 year ago

related to https://github.com/kaliiiiiiiiii/Selenium-Profiles/discussions/50

kaliiiiiiiiii commented 1 year ago

Implementation like in https://github.com/puppeteer/puppeteer/issues/3667#issuecomment-562280230 should be possible to fix this issue => working on it :)

kaliiiiiiiiii commented 11 months ago

might be possible to implement using something like

globalThis.tab_event = {checked_tabs:[],tab_callbacks:undefined}

tab_event.req_callback = async function(details) {
    var tab_id = details['tabId']
    if(tab_id >= 0){
        if(!(tab_event.checked_tabs.includes(tab_id))){
            await tab_event.tab_callbacks.forEach( async function (func){
                await func(details)
            })
            tab_event.checked_tabs.push(tab_id)
        }
    }
    return {};
  }

tab_event.on_tab_removed = function(tabId) {
    var idx= tab_event.checked_tabs.indexOf(tabId);
    tab_event.checked_tabs = tab_event.checked_tabs.splice(idx, 1);
}

tab_event.addEventListener = function(callback){
    if(!(tab_event.tab_callbacks)){
        tab_event.tab_callbacks = [callback]
        chrome.webRequest.onBeforeRequest.addListener(
          tab_event.req_callback,
          {urls: ["<all_urls>"]},
          ["blocking"]
        );
        chrome.tabs.onRemoved.addListener(tab_event.on_tab_removed)
    }else{tab_event.tab_callbacks.push(callback)}
}

tab_event.removeEventListener = function(listener){
    var idx= tab_event.tab_callbacks.indexOf(listener);
    tab_event.tab_callbacks = tab_event.tab_callbacks.splice(idx, 1);
}

tab_event.pause = async function(details){
    var tab_id = details['tabId']
    var target = {"tabId":tab_id}
    await chrome.debugger.attach(target,"1.2")
    await chrome.debugger.sendCommand(target,"Debugger.enable",{})
    chrome.debugger.sendCommand(target,"Debugger.pause",{},
        async function(){
            await chrome.tabs.executeScript(tab_id,{"code":"debugger"})
            await chrome.debugger.detach(target)
            }
        )
}

tab_event.continue_if_paused = async function(details){
    var tab_id = details['tabId']
    var target = {"tabId":tab_id}
    await chrome.debugger.attach(target,"1.2")
    await chrome.debugger.sendCommand(target,"Debugger.enable",{})
    await chrome.debugger.sendCommand(target,"Debugger.resume",{})

}

tab_event.addEventListener(console.log)
tab_event.addEventListener(tab_event.pause)

in selenium-injector

kaliiiiiiiiii commented 11 months ago

https://github.com/kaliiiiiiiiii/Selenium-Profiles/issues/58#issuecomment-1637178941

issue: cdp-requests don't get received while network resumed workaround: intercept network & inject debugger script

Fusseldieb commented 7 months ago

Yep, this issue still persists

Getting score F+

MarioCodarin commented 1 month ago

I solved matching the cpu cores of my pc with the cores in the profile.

from selenium_profiles.webdriver import Chrome
from selenium_profiles.profiles import profiles
from selenium.webdriver.common.by import By  # locate elements
from seleniumwire import webdriver
import multiprocessing

profile = profiles.Windows() # or .Android
profile['cdp']['cores'] = multiprocessing.cpu_count()
options = webdriver.ChromeOptions()
options.add_argument("--headless=new")
driver = Chrome(profile, options=options,
                uc_driver=False
                )
mehedihasanziku commented 1 month ago

I solved matching the cpu cores of my pc with the cores in the profile.

It’s already discussed above. we are looking for a way to bypass without match. because it’s easily fingerprinted.