FlareSolverr / FlareSolverr

Proxy server to bypass Cloudflare protection
MIT License
7.72k stars 660 forks source link

cf_clearance cookie from docker invalid #871

Closed y460699 closed 1 year ago

y460699 commented 1 year ago

Have you checked our README?

Have you followed our Troubleshooting?

Is there already an issue for your problem?

Have you checked the discussions?

Environment

- FlareSolverr version: latest
- Last working FlareSolverr version:
- Operating system: Windows, WSL2 linux for Docker
- Are you using Docker: yes
- FlareSolverr User-Agent (see log traces or / endpoint): 2023-08-22 12:38:41 INFO     FlareSolverr User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36
- Are you using a VPN: no
- Are you using a Proxy: no
- Are you using Captcha Solver: no
- If using captcha solver, which one:
- URL to test this issue: https://nowsecure.nl/

Description

Hello!

Running FlareSolver locally on my windows machine via the .exe works perfectly fine, the cf_clearance cookie and the user-agent can be used to avoid another cloudflare challenge. But as soon as I request through the docker container, it does solve it properly but the cookies received do not work for another request; gives me a cloudflare challenge. I might be stupid and have no idea if this is because of the network bridge in docker or if it's something else. Any ideas?

I'm using the python code from here to test; https://scrapeops.io/python-web-scraping-playbook/python-flaresolverr/#option-2-use-flaresolverr-to-only-retrieve-valid-cloudflare-cookies

Running it through docker, no success message:

$ docker run --name=flaresolverr -p 8191:8191 -e LOG_LEVEL=info --restart unless-stopped ghcr.io/flaresolverr/flaresolverr:latest 2023-08-22 12:56:49 INFO FlareSolverr 3.3.2 2023-08-22 12:56:49 INFO Testing web browser installation... 2023-08-22 12:56:49 INFO Platform: Linux-5.15.90.1-microsoft-standard-WSL2-x86_64-with-glibc2.31 2023-08-22 12:56:49 INFO Chrome / Chromium path: /usr/bin/chromium 2023-08-22 12:56:49 INFO Chrome / Chromium major version: 115 2023-08-22 12:56:49 INFO Launching web browser... 2023-08-22 12:56:50 INFO FlareSolverr User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36 2023-08-22 12:56:50 INFO Test successful! 2023-08-22 12:56:50 INFO Serving on http://0.0.0.0:8191 2023-08-22 12:57:08 INFO Incoming request => POST /v1 body: {'cmd': 'request.get', 'url': 'https://nowsecure.nl', 'maxTimeout': 60000} 2023-08-22 12:57:09 INFO Challenge detected. Title found: Just a moment... 2023-08-22 12:57:17 INFO Challenge solved! 2023-08-22 12:57:17 INFO Response in 8.828 s 2023-08-22 12:57:17 INFO 172.17.0.1 POST http://localhost:8191/v1 200 OK

Running it through the exe, I do get a success message:

2023-08-22 15:00:33 INFO FlareSolverr 3.3.2 2023-08-22 15:00:33 INFO Testing web browser installation... 2023-08-22 15:00:34 INFO Platform: Windows-10-10.0.22621-SP0 2023-08-22 15:00:34 INFO Chrome / Chromium path: C:\dev\nah\chrome\chrome.exe 2023-08-22 15:00:34 INFO Chrome / Chromium major version: 115 2023-08-22 15:00:34 INFO Launching web browser... 2023-08-22 15:00:34 INFO FlareSolverr User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36 2023-08-22 15:00:34 INFO Test successful! 2023-08-22 15:00:34 INFO Serving on http://0.0.0.0:8191 2023-08-22 15:00:40 INFO Incoming request => POST /v1 body: {'cmd': 'request.get', 'url': 'https://nowsecure.nl', 'maxTimeout': 60000} 2023-08-22 15:00:40 INFO Challenge detected. Title found: Just a moment... 2023-08-22 15:00:48 INFO Challenge solved! 2023-08-22 15:00:48 INFO Response in 8.713 s 2023-08-22 15:00:48 INFO 127.0.0.1 POST http://localhost:8191/v1 200 OK

Makes me wonder if it is indeed the docker bridge holding me back, I tried using --net=host but the request won't go through with that.

Logged Error Messages

-

Screenshots

No response

ilike2burnthing commented 1 year ago

Follow the troubleshooting steps here - https://github.com/FlareSolverr/FlareSolverr/wiki/Troubleshooting

Are you making the initial request from the host Windows machine or another container?