ultrafunkamsterdam / undetected-chromedriver

Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)
https://github.com/UltrafunkAmsterdam/undetected-chromedriver
GNU General Public License v3.0
9.68k stars 1.14k forks source link

Proxies too slow with undetected-chromedriver #1012

Open BukuBukuChagma opened 1 year ago

BukuBukuChagma commented 1 year ago

I have a script that uses proxies to visit a site. When using orignal selenium these proxies work fine and open the site but when I shift to undetected-chromedriver most of the proxies don't seem to work and the ones that does work are too slow that it's not worth working with. What would be the issue here? Am I doing something wrong or its bug on undetected-chromedriver side?

Code :

from selenium.webdriver.chrome.service import Service
from seleniumwire.undetected_chromedriver import webdriver

options = webdriver.ChromeOptions()
seloptions = {'proxy': {
                          'http': f'http://{ip_port}', 
                          'https':f'https://{ip_port}',
                          'socks4':f'socks4://{ip_port}',
                          'socks5':f'socks5://{ip_port}',
                          'no_proxy': 'localhost,127.0.0.1' # excludes
                      }  
                  }

options.add_argument("--log-level=3")
#options.add_argument('--proxy-server=%s' % ip_port)
options.add_argument("--mute-audio")
options.add_argument('--ignore-ssl-errors=yes')
options.add_argument('--ignore-certificate-errors')
options.binary_location = "C:\\Program Files\\Google\\Chrome Beta\\Application\\chrome.exe"
options.add_extension("Auto High Quality for YouTube™ 0.4.2.0.crx")

bot = webdriver.Chrome(service=Service("chromedriver.exe"), options=options, seleniumwire_options=seloptions)
strukiii commented 1 year ago

Mine is quite slow too, might be that you're being routed through somewhere on the other side of the world before getting to your final destination? My proxy provider is located in Singapore

BukuBukuChagma commented 1 year ago

Um then normal/orignal selenium should be slow too but it works fine. Its just when proxies are connected through this undetected-chromedriver they either become non-responsive or too slow.

On Thu, 2 Feb 2023, 4:35 am Carter Struk, @.***> wrote:

Mine is quite slow too, might be that you're being routed through somewhere on the other side of the world before getting to your final destination? My proxy provider is located in Singapore

— Reply to this email directly, view it on GitHub https://github.com/ultrafunkamsterdam/undetected-chromedriver/issues/1012#issuecomment-1412906892, or unsubscribe https://github.com/notifications/unsubscribe-auth/AYC3PGSS5WABMJ34LOAHYLDWVLXNNANCNFSM6AAAAAAUKCWTMI . You are receiving this because you authored the thread.Message ID: @.*** com>

ManiMozaffar commented 1 year ago

You have 3 options to connect a proxy to selenium

one is using --proxy-server. If you proxy include username or password, you may switch to IP auth. second one is to use an extension to connect to proxy, then load extension in option before loadup. and the third is the one as you're doing now, to use selenium wire.

I'm using first two options and none of them are slow. if you want to keep using selenium wire then you should open an issue there, not here. Using selenium wire is not a good practice for the purpose of this repo because website may detect you from the SSL handshake and SSL fingerprint.

in case, for second option:


import os

class ProxyExtension:
    manifest_json = """
    {
        "version": "1.0.0",
        "manifest_version": 2,
        "name": "Chrome Proxy",
        "permissions": [
            "proxy",
            "tabs",
            "unlimitedStorage",
            "storage",
            "<all_urls>",
            "webRequest",
            "webRequestBlocking"
        ],
        "background": {"scripts": ["background.js"]},
        "minimum_chrome_version": "76.0.0"
    }
    """

    background_js = """
    var config = {
        mode: "fixed_servers",
        rules: {
            singleProxy: {
                scheme: "http",
                host: "%s",
                port: %d
            },
            bypassList: ["localhost"]
        }
    };

    chrome.proxy.settings.set({value: config, scope: "regular"}, function() {});

    function callbackFn(details) {
        return {
            authCredentials: {
                username: "%s",
                password: "%s"
            }
        };
    }

    chrome.webRequest.onAuthRequired.addListener(
        callbackFn,
        { urls: ["<all_urls>"] },
        ['blocking']
    );
    """

    def __init__(self, path:str, host:str, port:str, user:str, password:str):
        parent_dir = f"{path}/"        
        self._dir = os.path.join(parent_dir, "extensions")

        try:
            os.mkdir(self._dir)
        except FileExistsError:
            pass

        try:
            manifest_file = os.path.join(self._dir, "manifest.json")

            with open(manifest_file, mode="w") as f:
                f.write(self.manifest_json)

            background_js = self.background_js % (host, port, user, password)
            background_file = os.path.join(self._dir, "background.js")
            with open(background_file, mode="w") as f:
                f.write(background_js)
        except FileNotFoundError:
            pass

    @property
    def directory(self):
        return self._dir

proxy_extension = ProxyExtension('path/to/somewhere','1.1.1.1','8888', 'username','password')
options.add_argument(f"--load-extension={proxy_extension.directory}")
ManiMozaffar commented 1 year ago

1031

Added proxy support to here, indeed if you need ;)

BukuBukuChagma commented 1 year ago

Thanks for the detailed explanation. Will try out the solutions you provided.

JackGod001 commented 1 year ago

1031 Added proxy support to here, indeed if you need ;)在此处添加了代理支持,确实如果您需要的话;)

I used your code but I visit google.com shows that this website cannot be accessed, sometimes it can, more than not, my proxy ip works on fingerprint browser software adspower.

donggoing commented 1 year ago

ProxyExtension seems undefined.