ultrafunkamsterdam / undetected-chromedriver

Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)
https://github.com/UltrafunkAmsterdam/undetected-chromedriver
GNU General Public License v3.0
9.07k stars 1.09k forks source link

Not bypassing cloudflare turnstile on VPS #1175

Open Thibault-collard opened 1 year ago

Thibault-collard commented 1 year ago

Hello, I use the 3.4.6 version with chrome 111 using the following params and it works great on my local computer: options = uc.ChromeOptions() driver = uc.Chrome(version_main=111,options=options,headless=True) driver.get("https://app.impact.com/login.user")

When i reproduce the code on my server, the cloudflare challenge not pass.

image- I try to connect to a different website using cloudflare but i had the same problem

DEBUG log: selenium.webdriver.remote.remote_connection:POST http://localhost:60447/session {"capabilities": {"firstMatch": [{}], "alwaysMatch": {"browserName": "chrome", "pageLoadStrategy": "normal", "goog:chromeOptions": {"extensions": [], "binary": "/usr/bin/google-chrome", "args": ["--remote-debugging-host=127.0.0.1", "--remote-debugging-port=36081", "--user-data-dir=/tmp/tmpsjubua5k", "--lang=en-US", "--no-default-browser-check", "--no-first-run", "--no-sandbox", "--test-type", "--window-size=1920,1080", "--start-maximized", "--no-sandbox", "--log-level=0"], "debuggerAddress": "127.0.0.1:36081"}}}} DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): localhost:60447

It sounds like my server is blocked by cloudflare for no reason. Does it already happened ? Do you think about something i didn't test yet ?

If you need more information, i would be happy to give you more details.

Thanks for your help !

mcastrogg commented 1 year ago

This is also happening to me on ChromeDriver 112.0.5615.49 (bd2a7bcb881c11e8cfe3078709382934e3916914-refs/branch-heads/5615@{#936})

masroore commented 1 year ago

Same issues here. The latest Chrome driver 112.0.* , cloudflare immediately blocks the requests. This didn't happen in driver version 111. Yet the same IP + user agent works fine when I visit the site via chrome and FF browsers.

NCLnclNCL commented 1 year ago

Me too

ultrafunkamsterdam commented 1 year ago

Running from a datacenter ip .. these ip blocks are all known by big tech .... Use a smaller or more local hosting party.

masroore commented 1 year ago

@ultrafunkamsterdam the DC IPs are probably not an issue - at least for me, as I can browse from the same IPs using a real browser. Also, playwright with stealth plugin seems to work fine for the moment (same set of IPv4s). Sure, these IPs are most likely held in poor reputation by CF, but it's happy to let real browsers pass... I've also verified access using an embedded CEF browser. No issues so far.

I'm using selenium-wire (to capture browser cookies & headers) which utilizes mitmproxy. I suspect CF is detecting the forward proxy by fingerprinting python's TLS handshake.

Google Chromium uses BoringSSL. cURL compiled with boringssl (instead of the default openssl) is known to pass cloudflare's TLS fingerprinting check. I am not aware of python bindings for boringssl.

At this point I haven't verified whether UC is being detected. My hunch is on mitmproxy being fingerprinted by cloudflares ML bots.

masroore commented 1 year ago

Update: I can confirm undetected_chromedriver is working fine. I've removed selenium-wire from my code; with plain undetected_chromedriver things are working as they should. It seems my issues had stemmed from CF detecting mitmproxy (possibly by fingerprinting TLS handshake).

No issues with DC IP addresses either. Some IPs may cause CF to raise the "verify-human" captcha test but doesn't outright block the request.

I've also created a test app in .net with embedded Chromium (cefsharp) - CF is happy to let it pass through.