ultrafunkamsterdam / undetected-chromedriver

Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)
https://github.com/UltrafunkAmsterdam/undetected-chromedriver
GNU General Public License v3.0
9.84k stars 1.15k forks source link

Chrome can not be opened #572

Open YOTYTeaM opened 2 years ago

YOTYTeaM commented 2 years ago

I get this error

patching driver executable C:\Users\YOTYTeaM\appdata\roaming\undetected_chromedriver\a1092ea028fa5cf8_chromedriver.exe
Exception in thread Thread-2:
Traceback (most recent call last):
  File "C:\Users\YOTYTeaM\AppData\Local\Programs\Python\Python39\lib\threading.py", line 954, in _bootstrap_inner
    self.run()
  File "C:\Users\YOTYTeaM\AppData\Local\Programs\Python\Python39\lib\threading.py", line 892, in run
    self._target(*self._args, **self._kwargs)
  File "G:\..\Basic.py", line 38, in create
    self.driver = self.make_driver()
  File "G:\..\Chrome.py", line 47, in make_driver
    driver = Chrome(options=option, browser_executable_path=os.getcwd()+"\\Resources\\Drivers\\chromedriver.exe")
  File "C:\Users\YOTYTeaM\AppData\Local\Programs\Python\Python39\lib\site-packages\undetected_chromedriver\__init__.py", line 401, in __init__
    super(Chrome, self).__init__(
  File "C:\Users\YOTYTeaM\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 70, in __init__
    super(WebDriver, self).__init__(DesiredCapabilities.CHROME['browserName'], "goog",
  File "C:\Users\YOTYTeaM\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\chromium\webdriver.py", line 93, in __init__
    RemoteWebDriver.__init__(
  File "C:\Users\YOTYTeaM\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 269, in __init__
    self.start_session(capabilities, browser_profile)
  File "C:\Users\YOTYTeaM\AppData\Local\Programs\Python\Python39\lib\site-packages\undetected_chromedriver\__init__.py", line 589, in start_session
    super(selenium.webdriver.chrome.webdriver.WebDriver, self).start_session(
  File "C:\Users\YOTYTeaM\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 360, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "C:\Users\YOTYTeaM\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 425, in execute
    self.error_handler.check_response(response)
  File "C:\Users\YOTYTeaM\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 247, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: cannot connect to chrome at 127.0.0.1:5456
from chrome not reachable
Stacktrace:
Backtrace:
        Ordinal0 [0x009A7413+2389011]
        Ordinal0 [0x00939F61+1941345]
        Ordinal0 [0x0082C520+836896]
        Ordinal0 [0x008205AB+787883]
        Ordinal0 [0x0084C128+966952]
        Ordinal0 [0x00845D9B+941467]
        Ordinal0 [0x00845B66+940902]
        Ordinal0 [0x00878C80+1150080]
        Ordinal0 [0x008788DA+1149146]
        Ordinal0 [0x00873F66+1130342]
        Ordinal0 [0x0084E546+976198]
        Ordinal0 [0x0084F456+980054]
        GetHandleVerifier [0x00B59632+1727522]
        GetHandleVerifier [0x00C0BA4D+2457661]
        GetHandleVerifier [0x00A3EB81+569713]
        GetHandleVerifier [0x00A3DD76+566118]
        Ordinal0 [0x00940B2B+1968939]
        Ordinal0 [0x00945988+1989000]
        Ordinal0 [0x00945A75+1989237]
        Ordinal0 [0x0094ECB1+2026673]
        BaseThreadInitThunk [0x7691FA29+25]
        RtlGetAppContainerNamedObjectPath [0x77507A9E+286]
        RtlGetAppContainerNamedObjectPath [0x77507A6E+238]
sebdelsol commented 2 years ago

howdy, how are we're supposed to reproduce that ? what's your options ? Are you launching drivers in multi thread ? what's your OS ? have you updated Selenium and undetected-chromedriver ? And please post some code to reproduce your issue.

varna9000 commented 2 years ago

Hi I have similar error: selenium.common.exceptions.WebDriverException: Message: unknown error: cannot connect to chrome at 127.0.0.1:35775 I'm just using the example code from the Readme. My path is /usr/bin/chromedriver and it's in the PATH.

sebdelsol commented 2 years ago

It doesn't help though : please show some code or at least the logs before the error. Btw which path do you mean ? driver_executable_path or browser_executable_path ? What's your undetected chromedriver version ?

varna9000 commented 2 years ago

Ok, so there's the code and error from the REPL:

Python 3.7.3 (default, Jan 22 2021, 20:04:44)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import undetected_chromedriver as uc
>>> driver = uc.Chrome()
^[[A^[[ATraceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/milen/nap/env/lib/python3.7/site-packages/undetected_chromedriver/__init__.py", line 408, in __init__
    keep_alive=keep_alive,
  File "/home/milen/nap/env/lib/python3.7/site-packages/selenium/webdriver/chrome/webdriver.py", line 73, in __init__
    service_log_path, service, keep_alive)
  File "/home/milen/nap/env/lib/python3.7/site-packages/selenium/webdriver/chromium/webdriver.py", line 99, in __init__
    options=options)
  File "/home/milen/nap/env/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 269, in __init__
    self.start_session(capabilities, browser_profile)
  File "/home/milen/nap/env/lib/python3.7/site-packages/undetected_chromedriver/__init__.py", line 590, in start_session
    capabilities, browser_profile
  File "/home/milen/nap/env/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 360, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/home/milen/nap/env/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 425, in execute
    self.error_handler.check_response(response)
  File "/home/milen/nap/env/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 247, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: cannot connect to chrome at 127.0.0.1:33107
from chrome not reachable
Stacktrace:
#0 0x5596f95fead3 <unknown>
#1 0x5596f935e3ff <unknown>
#2 0x5596f934cab9 <unknown>
#3 0x5596f9384f69 <unknown>
#4 0x5596f937cee6 <unknown>
#5 0x5596f93b82ba <unknown>
#6 0x5596f93b24e3 <unknown>
#7 0x5596f9387d1a <unknown>
#8 0x5596f9388e75 <unknown>
#9 0x5596f962cefd <unknown>
#10 0x5596f964619b <unknown>
#11 0x5596f962ec65 <unknown>
#12 0x5596f9646ec8 <unknown>
#13 0x5596f9622360 <unknown>
#14 0x5596f9662a68 <unknown>
#15 0x5596f9662be8 <unknown>
#16 0x5596f967c7fd <unknown>
#17 0x7f59f16a5fa3 <unknown>

ChromeDriver 90.0.4430.212 (e3cd97fc771b893b7fd1879196d1215b622c2bed-refs/branch-heads/4430@{#1429})

sebdelsol commented 2 years ago
varna9000 commented 2 years ago
varna9000 commented 2 years ago

Here's the same code with logging:

Python 3.7.3 (default, Jan 22 2021, 20:04:44)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import undetected_chromedriver as uc
>>> import logging
>>> logging.basicConfig(level=logging.DEBUG)
>>> driver = uc.Chrome()
DEBUG:undetected_chromedriver.patcher:getting release number from /LATEST_RELEASE
DEBUG:undetected_chromedriver.patcher:downloading from https://chromedriver.storage.googleapis.com/100.0.4896.60/chromedriver_linux64.zip
DEBUG:undetected_chromedriver.patcher:unzipping /tmp/tmp7r2xr2sb
INFO:undetected_chromedriver.patcher:patching driver executable /home/milen/.local/share/undetected_chromedriver/3ee16d33c206cfb6_chromedriver
DEBUG:selenium.webdriver.remote.remote_connection:POST http://localhost:47401/session {"capabilities": {"firstMatch": [{}], "alwaysMatch": {"browserName": "chrome", "pageLoadStrategy": "normal", "goog:chromeOptions": {"extensions": [], "binary": "/usr/bin/chromium", "args": ["--remote-debugging-host=127.0.0.1", "--remote-debugging-port=35373", "--user-data-dir=/tmp/tmpsi14ddi1", "--lang=bg-BG", "--no-default-browser-check", "--no-first-run", "--log-level=0"], "debuggerAddress": "127.0.0.1:35373"}}}, "desiredCapabilities": {"browserName": "chrome", "pageLoadStrategy": "normal", "goog:chromeOptions": {"extensions": [], "binary": "/usr/bin/chromium", "args": ["--remote-debugging-host=127.0.0.1", "--remote-debugging-port=35373", "--user-data-dir=/tmp/tmpsi14ddi1", "--lang=bg-BG", "--no-default-browser-check", "--no-first-run", "--log-level=0"], "debuggerAddress": "127.0.0.1:35373"}}}
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): localhost:47401
DEBUG:urllib3.connectionpool:http://localhost:47401 "POST /session HTTP/1.1" 500 769
DEBUG:selenium.webdriver.remote.remote_connection:Finished Request
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/milen/nap/env/lib/python3.7/site-packages/undetected_chromedriver/__init__.py", line 408, in __init__
    keep_alive=keep_alive,
  File "/home/milen/nap/env/lib/python3.7/site-packages/selenium/webdriver/chrome/webdriver.py", line 73, in __init__
    service_log_path, service, keep_alive)
  File "/home/milen/nap/env/lib/python3.7/site-packages/selenium/webdriver/chromium/webdriver.py", line 99, in __init__
    options=options)
  File "/home/milen/nap/env/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 269, in __init__
    self.start_session(capabilities, browser_profile)
  File "/home/milen/nap/env/lib/python3.7/site-packages/undetected_chromedriver/__init__.py", line 590, in start_session
    capabilities, browser_profile
  File "/home/milen/nap/env/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 360, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/home/milen/nap/env/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 425, in execute
    self.error_handler.check_response(response)
  File "/home/milen/nap/env/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 247, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: cannot connect to chrome at 127.0.0.1:35373
from chrome not reachable
Stacktrace:
#0 0x55b772617ad3 <unknown>
#1 0x55b7723773ff <unknown>
#2 0x55b772365ab9 <unknown>
#3 0x55b77239df69 <unknown>
#4 0x55b772395ee6 <unknown>
#5 0x55b7723d12ba <unknown>
#6 0x55b7723cb4e3 <unknown>
sebdelsol commented 2 years ago

Your Chrome and Chromedriver versions seems to match (and I guess Selenium would have explicitly said so otherwise?).

Yes Chrome is spawned in a detached process by undetected-chromedriver, if it fails Selenium throws this error because it can't connect to it. Maybe you've several Chrome versions installed on your system, does the Chromium found by find_chrome_executable() actually work fine ?

EDIT: So undetected_chromdriver actually uses the chromedriver 100 version and not the 90 one... Now please check the Chromium version found by the function above, you'll import it with :

from undetected_chromedriver import find_chrome_executable

EDIT2: please manually launch the Chromium in /usr/bin/chromium & check its version number.

varna9000 commented 2 years ago

Yes, it seem so:

import undetected_chromedriver
>>> undetected_chromedriver.find_chrome_executable()
'/usr/bin/chromium'

I don't think I have other Chromium versions.

varna9000 commented 2 years ago
Screenshot 2022-04-08 at 16 57 16
sebdelsol commented 2 years ago

Hence the error, chromedriver won't connect to a Chrome/Chromium with a different version number. It's surprising that Selenium didn't explicitly told you so in its error... Please update your Chromium and it should work.

EDIT: You should,

varna9000 commented 2 years ago

Ok Thank you. Is there a way to force undetected-chromdriver not to download the newest chrome or it defeats the purpose? Updating anything on Debian to the newest version is a nightmare.

sebdelsol commented 2 years ago

Yes you can

varna9000 commented 2 years ago

I tried with version_main=90 but get the same error, although undetected_chromedrive seems to be downloading the v.90 now. I'm really confused. Will try on another machine with up-to-date Chromium and revert.

>>> import logging
>>> logging.basicConfig(level=logging.DEBUG)
>>> driver = uc.Chrome(version_main=90)
DEBUG:undetected_chromedriver.patcher:getting release number from /LATEST_RELEASE_90
DEBUG:undetected_chromedriver.patcher:downloading from https://chromedriver.storage.googleapis.com/90.0.4430.24/chromedriver_linux64.zip
DEBUG:undetected_chromedriver.patcher:unzipping /tmp/tmp7rzrjxg0
INFO:undetected_chromedriver.patcher:patching driver executable /home/milen/.local/share/undetected_chromedriver/6255cd5679df8c48_chromedriver
DEBUG:selenium.webdriver.remote.remote_connection:POST http://localhost:55495/session {"capabilities": {"firstMatch": [{}], "alwaysMatch": {"browserName": "chrome", "pageLoadStrategy": "normal", "goog:chromeOptions": {"extensions": [], "binary": "/bin/chromium", "args": ["--remote-debugging-host=127.0.0.1", "--remote-debugging-port=32981", "--user-data-dir=/tmp/tmpvzlzd0rl", "--lang=bg-BG", "--no-default-browser-check", "--no-first-run", "--log-level=0"], "debuggerAddress": "127.0.0.1:32981"}}}, "desiredCapabilities": {"browserName": "chrome", "pageLoadStrategy": "normal", "goog:chromeOptions": {"extensions": [], "binary": "/bin/chromium", "args": ["--remote-debugging-host=127.0.0.1", "--remote-debugging-port=32981", "--user-data-dir=/tmp/tmpvzlzd0rl", "--lang=bg-BG", "--no-default-browser-check", "--no-first-run", "--log-level=0"], "debuggerAddress": "127.0.0.1:32981"}}}
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): localhost:55495
DEBUG:urllib3.connectionpool:http://localhost:55495 "POST /session HTTP/1.1" 500 183
DEBUG:selenium.webdriver.remote.remote_connection:Finished Request
DEBUG:undetected_chromedriver.patcher:successfully unlinked /home/milen/.local/share/undetected_chromedriver/68429eee05baf5bc_chromedriver
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/milen/nap/env/lib/python3.7/site-packages/undetected_chromedriver/__init__.py", line 408, in __init__
    keep_alive=keep_alive,
  File "/home/milen/nap/env/lib/python3.7/site-packages/selenium/webdriver/chrome/webdriver.py", line 73, in __init__
    service_log_path, service, keep_alive)
  File "/home/milen/nap/env/lib/python3.7/site-packages/selenium/webdriver/chromium/webdriver.py", line 99, in __init__
    options=options)
  File "/home/milen/nap/env/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 269, in __init__
    self.start_session(capabilities, browser_profile)
  File "/home/milen/nap/env/lib/python3.7/site-packages/undetected_chromedriver/__init__.py", line 590, in start_session
    capabilities, browser_profile
  File "/home/milen/nap/env/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 360, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/home/milen/nap/env/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 425, in execute
    self.error_handler.check_response(response)
  File "/home/milen/nap/env/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 247, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: cannot connect to chrome at 127.0.0.1:32981
from chrome not reachable
Stacktrace:
#0 0x55675d0c4e89 <unknown>

And passing driver_executable_path gives the following error:

>>> import undetected_chromedriver as uc
>>> import logging
>>> logging.basicConfig(level=logging.DEBUG)
>>> driver = uc.Chrome(driver_executable_path="/usr/bin/chromium")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/milen/nap/env/lib/python3.7/site-packages/undetected_chromedriver/__init__.py", line 408, in __init__
    keep_alive=keep_alive,
  File "/home/milen/nap/env/lib/python3.7/site-packages/selenium/webdriver/chrome/webdriver.py", line 73, in __init__
    service_log_path, service, keep_alive)
  File "/home/milen/nap/env/lib/python3.7/site-packages/selenium/webdriver/chromium/webdriver.py", line 90, in __init__
    self.service.start()
  File "/home/milen/nap/env/lib/python3.7/site-packages/selenium/webdriver/common/service.py", line 98, in start
    self.assert_process_still_running()
  File "/home/milen/nap/env/lib/python3.7/site-packages/selenium/webdriver/common/service.py", line 112, in assert_process_still_running
    % (self.path, return_code)
selenium.common.exceptions.WebDriverException: Message: Service /usr/bin/chromium unexpectedly exited. Status code was: 1
varna9000 commented 2 years ago

Ok, so this is strange. I've been executing the above code via ssh on my remote machine and the errors occur. However, when I logged in through Remote Desktop on that same machine and executed the code from Xterm (in the GUI), I see that Chrome is being started (window appears) and I can execute driver.get(url) without problem.

Is your library not meant to be run with headless browser?

sebdelsol commented 2 years ago

I’m not the author.

This package is not meant to be run headless except if you want to be easily detected : I’ve submitted a pull request to mitigate some obvious headless caveats but it’s far from working undetected, it was good enough for me though.

In your case It seems that Chrome is not started or crashes before the chromedriver can connect… Chrome is started as a detached process here.

I’m not familiar with ssh, maybe it messes up with some subprocess.Popen arguments ?

Please tell us what you’ve found out.