Getting "Retries exceeded the limit" message with error code 260 when using library

[carlos-mg89]

Hi!

First of all, I appreciate very much your efforts to make scrapping a Cloudflare based website possible.

My situation is the following one. I have setup an Android project and in the Main Activity, I'm trying to use your package as described in the main page.

So these are my steps:

1. Edit app gradle with: implementation 'com.zhkrb.cloudflare-scrape-android:scrape-webview:0.0.4' as described in the README
2. Add the following code in MainActivity (at the end of the onCreate() method):

val cloudFlare = Cloudflare(this, "https://www.autodoc.es/repuestos");
        cloudFlare.user_agent = "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:86.0) Gecko/20100101 Firefox/86.0";
        cloudFlare.setCfCallback(object : CfCallback {
            override
            fun onSuccess(
                cookieList: List<HttpCookie>,
                hasNewUrl: Boolean,
                newUrl: String
            ) {
                val check = true
            }

            override
            fun onFail(
                code: Int,
                msg: String
            ) {
                val check = false
            }
        })
        cloudFlare.getCookies()

I have added 2 breakpoints in the onSuccess() and in the onFail() methods, and the result is always the same. The code gets to the onFail() method, with code => 260 and msg => "Retries exceeded the limit".

Perhaps CloudFlare has done any changes recently? I can see the latest release of your library was done in February 2021, so quite recently.

Thanks in advance!

[zhkrb]

I'm checking.

You can try this demo test first, it works for my test

[zhkrb]

Oh, I know the problem. The cookie you set will make the website not return cf_clearance cookie.

Try to set mobile user-agent like Mozilla/5.0 (Linux; Android 6.0.1; SM-G920V Build/MMB29K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.98 Mobile Safari/537.36

[carlos-mg89]

Thanks for getting back :)

Do I have to add cf_clearance anywhere?

I've tried with the User Agent you've proposed, but cannot make it to the onSuccess callback...

[zhkrb]

no, cf_clearanceare required to bypass cf.

This looks like a problem with my insufficient test samples, but I don’t have time these days.

Will try to start fix bug on friday

[pixkk]

no, cf_clearanceare required to bypass cf.

This looks like a problem with my insufficient test samples, but I don’t have time these days.

Will try to start fix bug on friday

Hello, problem is still again, but changing user agent dont help (