codemanki / cloudscraper

--DEPRECATED -- 🛑 🛑 Node.js library to bypass cloudflare's anti-ddos page
MIT License
603 stars 141 forks source link

Cloudflare Code 1020 - Firewall Detection #261

Closed true-zero closed 5 years ago

true-zero commented 5 years ago

This website (https://sneakersnstuff.com) is some how detecting requests from cloudscraper using a custom firewall rule to block them.

After trying to investigate using Fiddler, my requests were being blocked but only if Fiddler was capturing traffic. The specific cause being "HTTPS Decryption" being active after taking a look at the available expressions for the firewall I've come to the conclusion that the rule involves the ssl boolean expression.

But I'm not sure where to go from here, since cloudscraper establishes an encrypted connection.

ghost commented 5 years ago

Hi @true-zero,

This website (https://sneakersnstuff.com) is some how detecting requests from cloudscraper using a custom firewall rule to block them. ... But I'm not sure where to go from here, since cloudscraper establishes an encrypted connection.

I don't have any problem accessing plain HTTP: http://sneakersnstuff.com

Cloudcraper doesn't force HTTPS. It reuses the protocol scheme of the current request's URI. So, if you're fetching e.g. http://example.com and that site doesn't redirect you to https://example, the connection will remain as plain text.

You can handle redirects manually, all of request's options are supported: https://github.com/request/request#requestoptions-callback

Cheers.

true-zero commented 5 years ago

I've tested that out and it works well for that specific URL but when I navigate to:

http://www.sneakersnstuff.com/en/2/sneakers?orderBy=Published&skip_layout=true&view_override=_ajax-filter

The request gets 1020'd again.

ghost commented 5 years ago

1020 ????

I get a reCaptcha when using my web browser... It's not a Cloudscraper problem, It's a reCaptcha.