search

lwthiker / curl-impersonate

curl-impersonate: A special build of curl that can impersonate Chrome & Firefox
MIT License
3.45k stars 229 forks source link

Bypassing Cloudflare #218

Open cvaclav opened 4 months ago

cvaclav commented 4 months ago

Hi,

I would like to ask you about your experience using curl-impersonate to fetch HTML content from site behind Cloudflare.

In fact, it worked well for us at first, but only for a few days. Then the 500 error page started coming back as response. We changed the browser from Chrome to FF, but again, that only worked for a few more days.

Are there any other options or settings to achieve this?

Thanks.

saajnn commented 3 months ago

same problem

phapntm commented 3 months ago

same problem right now...

lux5am commented 3 months ago

Maybe you request too often in short period of time?

phapntm commented 3 months ago

Maybe you request too often in short period of time?

Is that problem? I make many request but using proxy.

lux5am commented 3 months ago

Maybe you request too often in short period of time?

Is that problem? I make many request but using proxy.

yes. common problem. especially if you use proxy where it can be detected. the ip would be flagged as a bot. Some website uses stronger couldflare security setting. cloudflare wrap, nordvpn and other popular VPN already marked. just be moderate. use random sleep here and there

ukd1 commented 1 month ago

checkout FlareSolverr/FlareSolverr for that kind of thing