gildas-lormeau / single-file-cli

CLI tool for saving a faithful copy of a complete web page in a single HTML file (based on SingleFile)
GNU Affero General Public License v3.0
602 stars 63 forks source link

bypass cloudflare browser check #61

Open gemini0x2 opened 10 months ago

gemini0x2 commented 10 months ago

How can bypass the cloudflare browser check:

"Checking if the site connection is secure site.com needs to review the security of your connection before proceeding."

I know that chromedriver should be able to byapass this check, but I'm not sure what args to use. I have tried the following, but no luck:

I think the solution is in "browser-args", I tried changing the user agent to --user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3 and it worked, but then it crashes after crawling a few pages.

Is there a way to launch single-file with undetected-chromedriver

in-plaintext commented 9 months ago

example website and a command (Linux):

$ docker run singlefile 'https://www.phind.com/' > /tmp/phind.html

I don't know how to resolve the issue.

www.phind.com

Checking if the site connection is secure

www.phind.com needs to review the security of your connection before proceeding.
Enable JavaScript and cookies to continue

@gemini0x2 Blocking scripts certainly won't help with the access since the Cloudflare's check explicitly requires Javascript.

Klar commented 2 months ago

anyone got a solution for this?

/edit

I found a solution:

using the chrome extension: https://chromewebstore.google.com/detail/get-cookiestxt-locally/cclelndahbckbenkjhflpdbgdldlbecc to get my cookies.

and the cli parameter for cookie generated file: --browser-cookies-file