onesandzeroes-maker commented 2 years ago

Env: cygwin Python 3.9.10 Chrome version: 99.0.4844.82 Program: import undetected_chromedriver as uc driver = uc.Chrome()

Error:

import undetected_chromedriver.v2 as uc driver = uc.Chrome() Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python3.9/site-packages/undetected_chromedriver/init.py", line 233, in init patcher.auto() File "/usr/local/lib/python3.9/site-packages/undetected_chromedriver/patcher.py", line 130, in auto self.unzip_package(self.fetch_package()) File "/usr/local/lib/python3.9/site-packages/undetected_chromedriver/patcher.py", line 166, in fetch_package return urlretrieve(u)[0] File "/usr/lib/python3.9/urllib/request.py", line 239, in urlretrieve with contextlib.closing(urlopen(url, data)) as fp: File "/usr/lib/python3.9/urllib/request.py", line 214, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.9/urllib/request.py", line 523, in open response = meth(req, response) File "/usr/lib/python3.9/urllib/request.py", line 632, in http_response response = self.parent.error( File "/usr/lib/python3.9/urllib/request.py", line 561, in error return self._call_chain(args) File "/usr/lib/python3.9/urllib/request.py", line 494, in _call_chain result = func(args) File "/usr/lib/python3.9/urllib/request.py", line 641, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 404: Not Found

sebdelsol commented 2 years ago

Indeed this specific version isn't available whatever the OS. It's weird that https://chromedriver.storage.googleapis.com/LATEST_RELEASE returned this version number. I guess this has been resolved quickly.

onesandzeroes-maker commented 2 years ago

@sebdelsol just tried to reinstall the package through pip and still running into the same issue. It must be something on my end for such a straightforward use case to not work ;( Would appreciate any pointers

sebdelsol commented 2 years ago

What https://chromedriver.storage.googleapis.com/LATEST_RELEASE returns on your system?

onesandzeroes-maker commented 2 years ago

@sebdelsol sorry for the late reply...

$ curl https://chromedriver.storage.googleapis.com/LATEST_RELEASE 100.0.4896.60

$ python3.9 Python 3.9.10 (main, Jan 20 2022, 21:37:52) [GCC 11.2.0] on cygwin Type "help", "copyright", "credits" or "license" for more information.

import undetected_chromedriver as uc driver = uc.Chrome() Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python3.9/site-packages/undetected_chromedriver/init.py", line 233, in init patcher.auto() File "/usr/local/lib/python3.9/site-packages/undetected_chromedriver/patcher.py", line 130, in auto self.unzip_package(self.fetch_package()) File "/usr/local/lib/python3.9/site-packages/undetected_chromedriver/patcher.py", line 166, in fetch_package return urlretrieve(u)[0] File "/usr/lib/python3.9/urllib/request.py", line 239, in urlretrieve with contextlib.closing(urlopen(url, data)) as fp: File "/usr/lib/python3.9/urllib/request.py", line 214, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.9/urllib/request.py", line 523, in open response = meth(req, response) File "/usr/lib/python3.9/urllib/request.py", line 632, in http_response response = self.parent.error( File "/usr/lib/python3.9/urllib/request.py", line 561, in error return self._call_chain(args) File "/usr/lib/python3.9/urllib/request.py", line 494, in _call_chain result = func(args) File "/usr/lib/python3.9/urllib/request.py", line 641, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 404: Not Found

sebdelsol commented 2 years ago

Please retry your script with the logger set to debug logging.basicConfig(level=logging.DEBUG) and check what url is given in the "downloading from ..." message. What's the url ?
I assume you're on windows, so it should be : https://chromedriver.storage.googleapis.com/100.0.4896.60/chromedriver_win32.zip can you download the file from this url in your regular Chrome browser ?

Does that works ?

from urllib.request import urlretrieve
u = "https://chromedriver.storage.googleapis.com/100.0.4896.60/chromedriver_win32.zip"
print(len(urlretrieve(u)[0]))

onesandzeroes-maker commented 2 years ago

@sebdelsol great troubleshooting! The question is how to set %s in cygwin environment? For whatever reason, it is not set...

import logging logging.basicConfig(level=logging.DEBUG) import undetected_chromedriver as uc driver = uc.Chrome() DEBUG:undetected_chromedriver.patcher:getting release number from /LATEST_RELEASE DEBUG:undetectedchromedriver.patcher:downloading from https://chromedriver.storage.googleapis.com/101.0.4951.41/chromedriver%s.zip

sebdelsol commented 2 years ago

Sorry I didn't see you were on Cygwin and I'm not familiar with it... I'm really not sure it'll works on Cygwin. I guess you have a built of Chromium for it ?

Anyway on your system sys.platform returns "cygwin" and the Patcher class doesn't handle this case : e.g. Patcher.zip_name isn't formatted and stays set as "chromedriver_%s.zip" hence the weird error...

sys.platform is checked in several other places but it seems to be an issue only in patcher.py (if you don't use the old undetected_chromedriver._compat).

So you can fork undetected-chromedriver and try to modify it : Replace this line here and this other one here with if platform.endswith(("linux", "cygwin")): .

Feel free to create a pull request for this.

CharlieSalsa commented 8 months ago

I solved it using an older version of chrome. 😁 I don't know how but it is solved

browser = uc.Chrome(version_main = 120) Chrome version: 121.0.6167.162

lupeo commented 8 months ago

I solved it using an older version of chrome. 😁 I don't know how but it is solved

browser = uc.Chrome(version_main = 120) Chrome version: 121.0.6167.162

Thank you, it worked for me

omar-avila commented 8 months ago

Did everyone just spontaneously get this error from undetected chromedriver? I'm running this in a docker container which should be stable given it's the same image, but wondering how I can set this up to prevent it from spontaneously breaking like this?

I solved it using an older version of chrome. 😁 I don't know how but it is solved

browser = uc.Chrome(version_main = 120) Chrome version: 121.0.6167.162

larryhopecode commented 8 months ago

Wow. this issue is troubling me for three hours until I found the solution from you guys. Thanks alot

lyujw commented 8 months ago

I solved it using an older version of chrome. 😁 I don't know how but it is solved

browser = uc.Chrome(version_main = 120) Chrome version: 121.0.6167.162

Legend!

LucaL1fe commented 8 months ago

Although Thanks by me, seems i am not the only one with this new Error without changing anything 😅👌

denixyd commented 8 months ago

It seems the zip for the most recent chromedriver update is not available in the domain that uc uses "https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing" but it is available in the default googleapis storage "https://storage.googleapis.com/chrome-for-testing-public".

editing the "venv\Lib\site-packages\undetected_chromedriver\patcher.py" sorts out the issue. Although I have done it locally. Below is the edit that I have made in the 'fetch_package' function found in uc "patcher.py" file. I have adjusted the "download_url" variable.

def fetch_package(self):
        """
        Downloads ChromeDriver from source

        :return: path to downloaded file
        """
        zip_name = f"chromedriver_{self.platform_name}.zip"
        if self.is_old_chromedriver:
            download_url = "%s/%s/%s" % (self.url_repo, self.version_full.vstring, zip_name)
        else:
            zip_name = zip_name.replace("_", "-", 1)
            # download_url = "https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/%s/%s/%s"
            download_url = "https://storage.googleapis.com/chrome-for-testing-public/%s/%s/%s"
            download_url %= (self.version_full.vstring, self.platform_name, zip_name)

        logger.debug("downloading from %s" % download_url)
        return urlretrieve(download_url)[0]

Seckna commented 8 months ago

It seems the zip for the most recent chromedriver update is not available in the domain that uc uses "https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing" but it is available in the default googleapis storage "https://storage.googleapis.com/chrome-for-testing-public".

editing the "venv\Lib\site-packages\undetected_chromedriver\patcher.py" sorts out the issue. Although I have done it locally. Below is the edit that I have made in the 'fetch_package' function found in uc "patcher.py" file. I have adjusted the "download_url" variable.
def fetch_package(self):
        """
        Downloads ChromeDriver from source

        :return: path to downloaded file
        """
        zip_name = f"chromedriver_{self.platform_name}.zip"
        if self.is_old_chromedriver:
            download_url = "%s/%s/%s" % (self.url_repo, self.version_full.vstring, zip_name)
        else:
            zip_name = zip_name.replace("_", "-", 1)
            # download_url = "https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/%s/%s/%s"
            download_url = "https://storage.googleapis.com/chrome-for-testing-public/%s/%s/%s"
            download_url %= (self.version_full.vstring, self.platform_name, zip_name)

        logger.debug("downloading from %s" % download_url)
        return urlretrieve(download_url)[0]

Thanks bro it's working for me can you tell me where did you find this link "https://storage.googleapis.com/chrome-for-testing-public/%s/%s/%s"?

ilike2burnthing commented 8 months ago

Issue resolved in v3.5.5

autpal commented 8 months ago

FIXED -------Upgrade your UC to version 3.5.5

pip install --upgrade undetected-chromedriver

javakam commented 4 months ago

不推荐只能临时解决问题(Only temporary solutions) driver = uc.Chrome(version_main=125, options=options)

推荐(Recommended)


from webdriver_manager.chrome import ChromeDriverManager
chromedriver_path = ChromeDriverManager().install()

import undetected_chromedriver as uc from selenium.webdriver.support.ui import WebDriverWait

usage

driver = uc.Chrome(options=options, driver_executable_path=chromedriver_path) ...


from   [https://github.com/ultrafunkamsterdam/undetected-chromedriver/issues/1904#issuecomment-2146739297](https://github.com/ultrafunkamsterdam/undetected-chromedriver/issues/1904#issuecomment-2146739297)

ultrafunkamsterdam / undetected-chromedriver

urllib.error.HTTPError: HTTP Error 404: Not Found #565

usage