Struggle with cloudflare when using headless option

ultrafunkamsterdam / undetected-chromedriver

Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)

https://github.com/UltrafunkAmsterdam/undetected-chromedriver

GNU General Public License v3.0

10.08k stars 1.17k forks source link

Struggle with cloudflare when using headless option #390

Open RajPorus19 opened 2 years ago

RajPorus19 commented 2 years ago

For some reason, if I try to use the headless option, undected-chromedriver doesn't get on the page at all, even with waiting past 10 seconds on it. Has anyone managed to scrap a website under cloudflare protection while being in headless mode ? Any help would be greatly appreciated ! For some extra info, I am trying to scrape this website : https://ettv.unblockit.tv/ It's for a school project where I do charts for torrents popularity. Thanks in advance for you awnsers :)

iamMeshi commented 2 years ago

You currently can't get passed Cloudflare whilst using --headless mode see here

y-tai commented 2 years ago

The answer is do not use --headless. I think that headless mode is not the only way that use undected-chromedriver without UI display, you can try another solution if you just want to run it in the command line.

RajPorus19 commented 2 years ago

@tyIceStream so how would you proceed to make it work without UI ? In my case I need it to run inside a docker container

y-tai commented 2 years ago

You can try this solution. In my case I run it in Github Action, not similar with yours.

Obie-Wan commented 2 years ago

now its blocked by cloudflare even wo headless