Using cloudscraper to prevent Cloudflare's bans

[caribpa]

cloudscraper is a module that attempts to bypass Cloudflare's anti-bot defenses for webcrawling/scraping purposes.

For sqlmap it may allow the use of a bigger --threads value for those pages under Cloudflare, among maybe other things, but honestly I am dreaming of a best case scenario. The worst that can happen is that it does nothing to prevent the bans and ends up being slower than before.

Unfortunately this won't be a trivial change as it would be like going down the same path to replace urllib with Requests as cloudscraper is built on top of it.

It could also be added as an optional dependency for backwards compatibility/minimal dependency build, but the work would be even more.

Is there any interest in this to begin with?

[ghost]

@Caribpa I would love to see this

[stamparm]

Can anybody explain in simple terms what are the benefits of cloudscraper except dumb extraction of the cookie (sqlmap already does this). Just as an example of the failed run of the exact thing that YOU expect it to do:

[stamparm]

1) My latest amusement is following along as some hackers are trying to get around @Cloudflare ’s bot mitigation. They imagine we’re furiously making code changes. In reality, our AI systems detect their changes and auto adapt. And they pull their hair out (https://twitter.com/eastdakota/status/1257150545623568385?s=21) 2) Cloudflare changed? aka "Detected the new Cloudflare challenge" #209 (Opened) (https://github.com/VeNoMouS/cloudscraper/issues/209)

[caribpa]

@stamparm the idea of using cloudscraper is to delegate the task of bypassing Cloudflare to a dedicated library so that sqlmap developers may focus on different tasks.

This said I agree that currently sqlmap is able to access sites under Cloudflare, but that may change in the future (the same can be said about cloudscraper though).

The other thing I want to point out is that using sqlmap on sites under Cloudflare may lead to bans (after some conditions) whereas this may not be true (now or in the future) with cloudscraper.

I haven't tested cloudscraper on such edge cases, and I know that talking about what may change in the future is not a good point, but I wanted to know if other people here have had a great experience with cloudscraper when accessing sites under Cloudflare so that it made sense integrating it into sqlmap.

For now I'm keeping an eye on cloudscraper and I'll report it here if I find it capable of accessing a site that sqlmap is unable to.

PS: Why should you feel discouraged about what the Cloudflare's people say of their anti-bot system, let it be true? Of course they'd say what they have to, but AFAIK history has proven such people wrong, so take it like a grain of salt and let's keep trying, shall we? 😉

[stamparm]

"Why should you feel discouraged about what the Cloudflare's people say of their anti-bot system, let it be true?" <- please take a good luck into my comment at https://github.com/sqlmapproject/sqlmap/issues/4147#issuecomment-648679769.

Currently, cloudscraper is useless (which I proved at the referenced comment above). I even posted an opened Issue at the project itself which just proves my point

Basically, implementing a support in sqlmap for a library which currently does nothing, while promising people that it actually does something is a NO GO

p.s. I am not against implementing nor supporting the cloudscraper. Only thing is that currently I don't see a point in it

[caribpa]

When I replied to you I understood completely what you wrote, in fact I agreed with you that currently cloudscraper doesn't do anything special (although I haven't investigated if it is able to bypass behavior that lead to bans). I never said that it should be integrated in its current state (if ever), I invite you to re-read my comment for further clarification of what I meant.

[stamparm]

Perfectly understood. Closing this down because it doesn't have any purpose this moment