kaliiiiiiiiii / Selenium-Profiles

undetected Selenium using chromedriver and emulation / device profiles
Other
255 stars 29 forks source link

Headless bug in undetected-chromedriver #45

Closed anilabhadatta closed 1 year ago

anilabhadatta commented 1 year ago
profile = profiles.Android()
mydriver = Chrome(
            profile, options=options, uc_driver=True,
            executable_path=f"./chromedrivers/chromedriver.exe")
if headless:
     mydriver.options.add_argument("--headless=new")
driver = mydriver.start()

This is my code to start the browser in headless mode but it is not working, In Non headless mode the code is working as expected.

Error: '<' not supported between instances of 'NoneType' and 'int'

As of now i know that i am getting this error when i am using uc_driver=True. There is a bug in official undetected_chromedriver where i need to write the following code to initiate chrome as headless mode using UC driver.

driver = ucdriver.Chrome(headless=headless,  version_main=112,
                                 options=options, driver_executable_path=f"./chromedrivers/chromedriver.exe")

I need to put version_main and headless=True so that chrome can load in headless mode using ucdriver. If i didn't apply version_main as parameter then i was receiving the same error.

Can you add headless and version_main to your selenium_profiles so that ucdriver can load properly?

anilabhadatta commented 1 year ago

I have made a fix, willing to share with you.

kaliiiiiiiiii commented 1 year ago

Thanks for submitting this issue. I've thought from the beginning on about passing the arguments directly to undetected-chromedriver, but assumed it should also work without, as he checks at undetected_chromedriver/init.py#L290 for the --headless key. Wonder what exactly caused this bug.

I'll have a look in the.. probably next days :)

@anilabhadatta What version of undetected-chromedriver are you running on btw?

UPDATE: Guess what's basically happening is https://github.com/ultrafunkamsterdam/undetected-chromedriver/issues/1200 ==> headless doesn't need to get passed directly, but version_main needs to be int, and can't be None.

UPDATE-2 This issue only persists when:

anilabhadatta commented 1 year ago

@kaliiiiiiiiii the latest version of undetected chromedriver is installed. What i have seen is i need to pass version_main as parameter along with headless=True.

The fix that you can apply is allowing two parameters in your webdriver.py init function i.e headless and version_main And after line 82 you can add the following lines.

self.kwargs.update({"headless":self.headless})

self.kwargs.update({"version_main":self.version_main})

And while calling the init function, pass headless=True and version_main=(int) This should technically fix the situation.

I didn't apply version_main in your code since i actually removed that parameter from undetected chromedriver itself but it not a good fix since chrome version less than 108 will cause errors.

Try the above solution, it should fix. I could create a pull request if you want but i need some time.

kaliiiiiiiiii commented 1 year ago

Yeah thought about something like that. Althought I'd preferably not add 2 arguments to my driver just for fixing a bug in undetected-chromedriver:)

I guess I'll add automatic detection for the Chrome version and headless options, since we don't want any inconsistenciesđŸ‘€

anilabhadatta commented 1 year ago

@kaliiiiiiiiii yes that would be better, i think getting chrome version will he be easy. Maybe using os library you can probably call the chrome at specific or default path and pass a parameter as version, which may return the version of chrome.

As for headless mode you are already taking headless mode as chromeoptions so that can be implemented easily in yout init.py

kaliiiiiiiiii commented 1 year ago

getting chrome version will he be easy.

Uhh not as easy as thought:) On linux, chrome --version works cleanly, but Windows has some hard times:

Isssue with those: Probably not reliable

paamayim commented 1 year ago

On linux, chrome --version works cleanly, but Windows has some hard times:

  • chrome.exe includes the version at some point, but reading & parsing those files takes up processing power, time, and RAM -_-
  • getting some REGISTRY key works as well.

It looks like that's what chromedriver-autoinstaller did here

import chromedriver_autoinstaller

version_main = int(chromedriver_autoinstaller.get_chrome_version().split(".")[0])
this.driver = webdriveruc.Chrome(options=options, seleniumwire_options=seleniumwire_options, version_main=version_main)
kaliiiiiiiiii commented 1 year ago

It looks like that's what chromedriver-autoinstaller did here

import chromedriver_autoinstaller

version_main = int(chromedriver_autoinstaller.get_chrome_version().split(".")[0])
this.driver = webdriveruc.Chrome(options=options, seleniumwire_options=seleniumwire_options, version_main=version_main)

@paamayim Thanks a lot! Didn't test it yet, but looks great

I might copy only the get_chrome_version() function and put some credits in there for implementation.

kaliiiiiiiiii commented 1 year ago

fix seems to be indended by undetected-chromedriver (see commit ) closing therefore