search

ultrafunkamsterdam / undetected-chromedriver

Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)
https://github.com/UltrafunkAmsterdam/undetected-chromedriver
GNU General Public License v3.0
9.84k stars 1.15k forks source link

Blocked by DataDome on Footlocker #1336

Open safnak opened 1 year ago

safnak commented 1 year ago

Hi, Please attempt to use UC on https://www.footlocker.co.uk/ and you'll notice that after a couple of clicks, it gets instantly blocked, or the captcha comes up and then it gets blocked regardless of the captcha result.

Any solution would be much appreciated 👍

lonewolf-github commented 1 year ago

If you don't get instantly blocked by a service, but after some clicks, its usually a sign that your behaviour is whats causing the detection. Try pausing more and generally behaving like a human user would

safnak commented 1 year ago

If you don't get instantly blocked by a service, but after some clicks, its usually a sign that your behaviour is whats causing the detection. Try pausing more and generally behaving like a human user would

All I'm trying to do is standard human interaction such as logging into my account. Pausing doesn't seem to help

MataloDeEiras commented 1 year ago

I'm getting blocked by datadome/cloudfare on another site too. Issue seems to be that the undected chromedriver browser is being detected, logging into the site manually from normal chrome is fine. Trying to log in manually from an undected chromdriver instance is blocked too, so it can't be about speed/human interaction.

Disclaimer: I'm using seleniumbase with undetected chromedriver mode, but I am pretty sure it uses undected chromedriver.

safnak commented 1 year ago

I'm getting blocked by datadome/cloudfare on another site too. Issue seems to be that the undected chromedriver browser is being detected, logging into the site manually from normal chrome is fine. Trying to log in manually from an undected chromdriver instance is blocked too, so it can't be about speed/human interaction.

Disclaimer: I'm using seleniumbase with undetected chromedriver mode, but I am pretty sure it uses undected chromedriver.

Yes, this is the exact same issue I'm encountering. I've been looking for a solution for a while but no luck 😕

Inocious commented 1 year ago

Same here different website using datadome. If I'm editing uc "init.py" to launch chrome only with flags "--remote-debugging-host=127.0.0.1 --remote-debugging-port=XXXXX" with uc and navigating to datadome protected website manually, I'm getting instant ban. (chrome is then being called ""C:\Program Files\Google\Chrome\Application\chrome.exe" --remote-debugging-host=127.0.0.1 --remote-debugging-port=63452 "--user-data-dir=C:\Users\XXXXX\AppData\Local\Google\Chrome\User Data")

If I'm removing those two flags but still using undetected_chromedriver to start browser, Datadome doesn't block me and let me pass. (Chrome is then being called "C:\Program Files\Google\Chrome\Application\chrome.exe" "--user-data-dir=C:\Users\XXXXX\AppData\Local\Google\Chrome\User Data")

If I'm opening chrome manually with "C:\Program Files\Google\Chrome\Application\chrome.exe" --remote-debugging-host=127.0.0.1 --remote-debugging-port=63452 "--user-data-dir=C:\Users\XXXXX\AppData\Local\Google\Chrome\User Data" datadome let me pass.

safnak commented 1 year ago

Same here different website using datadome. If I'm editing uc "init.py" to launch chrome only with flags "--remote-debugging-host=127.0.0.1 --remote-debugging-port=XXXXX" with uc and navigating to datadome protected website manually, I'm getting instant ban. (chrome is then being called ""C:\Program Files\Google\Chrome\Application\chrome.exe" --remote-debugging-host=127.0.0.1 --remote-debugging-port=63452 "--user-data-dir=C:\Users\XXXXX\AppData\Local\Google\Chrome\User Data")

If I'm removing those two flags but still using undetected_chromedriver to start browser, Datadome doesn't block me and let me pass. (Chrome is then being called "C:\Program Files\Google\Chrome\Application\chrome.exe" "--user-data-dir=C:\Users\XXXXX\AppData\Local\Google\Chrome\User Data")

If I'm opening chrome manually with "C:\Program Files\Google\Chrome\Application\chrome.exe" --remote-debugging-host=127.0.0.1 --remote-debugging-port=63452 "--user-data-dir=C:\Users\XXXXX\AppData\Local\Google\Chrome\User Data" datadome let me pass.

Does this solution allow you to run multiple instances of uc browser and if you get blocked once then are you able to start a new session which is unblocked?

EDIT: just tried the above solution and the browser starts up but then won't do anything further afterwards such as navigating to a given page

Inocious commented 1 year ago

Yes of course, removing --remote-debugging-host=127.0.0.1 --remote-debugging-port=XXXXX prevent uc from communicating with the browser. I tried it just as test purposes, I then used opened browser manually.

As of today there is no way to use uc with datadome. It seems detecting uc in combination with --remote-debugging-host=127.0.0.1 --remote-debugging-port=XXXXX flags.

My versions: Chrome browser: driver.capabilities['browserVersion'] == v114.0.5735.110 (same behavior with v114.0.5735.134) Chromedriver: driver.capabilities['chrome']['chromedriverVersion'].split(' ')[0] == v114.0.5735.90

EDIT: Seems the switch "--remote-debugging-port=XXXXX" combined with uc triggers the datadome ban. So when uc calls browser with ""C:\Program Files\Google\Chrome\Application\chrome.exe" --remote-debugging-port=XXXXX"--user-data-dir=C:\Users\YYYYY\AppData\Local\Google\Chrome\User Data"" it triggers datadome ban.

So as a conclusion: Switch "--remote-debugging-host=127.0.0.1" used lonely with uc doesn't trigger datadome ban. But "--remote-debugging-port" used with uc does trigger datadome ban.

safnak commented 1 year ago

Yes of course, removing --remote-debugging-host=127.0.0.1 --remote-debugging-port=XXXXX prevent uc from communicating with the browser. I tried it just as test purposes, I then used opened browser manually.

As of today there is no way to use uc with datadome. It seems detecting uc in combination with --remote-debugging-host=127.0.0.1 --remote-debugging-port=XXXXX flags.

My versions: Chrome browser: driver.capabilities['browserVersion'] == v114.0.5735.110 (same behavior with v114.0.5735.134) Chromedriver: driver.capabilities['chrome']['chromedriverVersion'].split(' ')[0] == v114.0.5735.90

EDIT: Seems the switch "--remote-debugging-port=XXXXX" combined with uc triggers the datadome ban. So when uc calls browser with ""C:\Program Files\Google\Chrome\Application\chrome.exe" --remote-debugging-port=XXXXX"--user-data-dir=C:\Users\YYYYY\AppData\Local\Google\Chrome\User Data"" it triggers datadome ban.

So as a conclusion: Switch "--remote-debugging-host=127.0.0.1" used lonely with uc doesn't trigger datadome ban. But "--remote-debugging-port" used with uc does trigger datadome ban.

Ahhh okay i understand you now. @ultrafunkamsterdam any possible solution for this?

Also, is there any way to prevent the browser from automatically closing down when removing "--remote-debugging-port=XXXXX"? try except doesn't work.

Inocious commented 1 year ago

I think you're missing something here. Those two args are mandatory for uc to interact with Chrome browser. UC has to interact with your browser to send instruction to execute. Those two arg that uc is passing "--remote-debugging-host=127.0.0.1 --remote-debugging-port=XXXXX" is like the communication channel between uc and chrome browser. There is no way to bypass those two args. Which seems to be the issue here as datadome seems to detect somehow the arg --remote-debugging-port=XXXX in combination with the use of UC.

safnak commented 1 year ago

I think you're missing something here. Those two args are mandatory for uc to interact with Chrome browser. UC has to interact with your browser to send instruction to execute. Those two arg that uc is passing "--remote-debugging-host=127.0.0.1 --remote-debugging-port=XXXXX" is like the communication channel between uc and chrome browser. There is no way to bypass those two args. Which seems to be the issue here as datadome seems to detect somehow the arg --remote-debugging-port=XXXX in combination with the use of UC.

Apologies yes I misunderstood on how it worked.

The only thing I can think of right now which might work is to get datadome cookies from elsewhere somehow and then use them in my UC browser. Need an efficient way of doing this.

Inocious commented 1 year ago

No worries :) Yep same. Seems really strange website shouldn't be able to detect command line passed args so I'm curious about how this could trigger the datadome ban. The use of --remote-debugging-port arg in combination of uc probably triggers some unknown flags that datadome is checking thnaks to JS. Needs to find which ones now to fix them.

safnak commented 1 year ago

No worries :) Yep same. Seems really strange website shouldn't be able to detect command line passed args so I'm curious about how this could trigger the datadome ban. The use of --remote-debugging-port arg in combination of uc probably triggers some unknown flags that datadome is checking thnaks to JS. Needs to find which ones now to fix them.

I don't have much knowledge on the system so gonna be difficult for me to debug :(

Let's hope we get a fix soon 🤞