KyranRana / cloudflare-bypass

A new and improved PHP library which bypasses the Cloudflare IUAM page using cURL
MIT License
275 stars 98 forks source link

'[ERROR] Captcha (retry: 1) -> not supported! Unknown error -> Captcha page is not supported!' or ' [ERROR] UAM (retry: 0) -> clearance cookie missing!' #180

Open nitemare opened 4 years ago

nitemare commented 4 years ago

I'm still getting errors that have been reported in issues that were closed. I am using the latest commit available. I have installed with composer, as well as downloaded the zip and copied the files over the composer-version files.

Issue Depending only on the HTTP Headers used, I am getting one of 2 errors, and I cannot successfully retrieve the site. Those errors are:

[ERROR] Captcha (retry: 1) -> not supported! Unknown error -> Captcha page is not supported!

or

[ERROR] UAM (retry: 0) -> clearance cookie missing!! Unknown error -> CF clearance could not be found!

See below for full output and setup

Scenario 1

To test this out, I have been using the CFCurlTest.php example. (see details below for example code)

require "{Home Path}/libraries/vendor/autoload.php";

use CloudflareBypass\CFCurlImpl;
use CloudflareBypass\Model\UAMOptions;

$url = "https://torrentz2.eu/";
$ch = curl_init($url);

// Want to cache clearance cookies ?
curl_setopt($ch, CURLOPT_COOKIEJAR, "cookies.txt");
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookies.txt");

curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLINFO_HEADER_OUT, true);
curl_setopt($ch, CURLOPT_HTTPHEADER,
    array(
        "Upgrade-Insecure-Requests: 1",
        "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36",
        "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3",
        "Accept-Language: en-US,en;q=0.9"

$cfCurl = new CFCurlImpl();

$cfOptions = new UAMOptions();
$cfOptions->setVerbose(true);
// $cfOptions->setDelay(5);

try {
    $page = $cfCurl->exec($ch, $cfOptions);

    // Want to get clearance cookies ?
    $cookies = curl_getinfo($ch, CURLINFO_COOKIELIST);

} catch (ErrorException $ex) {
    echo "Unknown error -> " . $ex->getMessage();
}

If I use it as-is, only changing the url to:

$url = "https://torrentz2.eu/";

and using the existing headers (see below details)

curl_setopt($ch, CURLOPT_HTTPHEADER,
    array(
        "Upgrade-Insecure-Requests: 1",
        "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36",
        "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3",
        "Accept-Language: en-US,en;q=0.9"
    ));

Results of Scenario 1 This will return NULL for $page and the Verbose error is:

[ERROR] Captcha (retry: 1) -> not supported! Unknown error -> Captcha page is not supported!

See details below for Verbose Output

[INFO] UAM (retry: 0) -> applying HTTPS settings 
[INFO] UAM (retry: 0) -> waiting for 5 seconds! 
[INFO] UAM (retry: 0) -> getting form params... 
[INFO] UAM (retry: 0) -> (r param: 805512f3ffefd8f715fa26aefac9de54aec0ae2a-1585084265-0-Aa0PuyGqiatYEAp4U96bwt8qR0AfSblp3Q3iaLdySFZqKQeg/hVtP5+PQ0heCF8687J6eXOQ8QUA4uWJjcw0hhld1PBWuqcz2BWc0jIktmOHIHqzynfqHjZr1bxnrwF2Z/W1grihpT0Ef2GEb0kk9e8ufit8C2zZwyaT5Dbvgg+iD1dGIrqRGjEcSo9rTmFYUfvJxBIQMw6MPIPcZDYafoJcUNgTHJ+/Jt2sQFxfQnEsGV1LPVfXgbPjXOgy4Cq1pxbHGgE/Ys84db3UzuSHxODzUwxRo6k6EIT3GUucCB+EomL4nwQxeXCGG88epNU4IYdCMBNs/4lCueB8vgcWac+bRzpcABrKEkXX6Xpq1pn4RHoD1L2ltgtIGsP5xhgnQATtZl3nNoHA5WzXyOIww6bkoXUeyVHBBrcVtRYzDnzKUogU9LqCxmBIGf9dNmCai6jzJlXAyG0bB1omN6Z47ahF5TgY3gnywH3ZWrzDU/4N7Qb6aaNqKWX6SLsFIRsqIpDFB/FWRAI0hxn21o49ClLl3yDTotSeMBx0iJh+FrH58LVpIQ91ANH6Tej2D6BlNHs1p6tGeDd+GRwabUrpP2xBvW0lj2Kbw8rGE7qOGFoNCy6SiNd6QBK9gxGpqJPqRHyRPZ4cTxvOXAH3bsk8zkFE+qiP1cdJ774R1b2FM28AmEq74sONy+LOLMkhHnU7YuRQeIH7T/8drR3wGlCx/lcQKQuo61K/4F1DRtvjgvyCkOFKr5qjACbRMdd66nb9MzW3wv2vgSuZWZ+Tfvy4LoRBtC41v00LdY+tXqBAkIVQ5pbkcduES9AHXjWPscbFRh5WJGFlc+xhpFbsdzGPgyB5/y1i+VHRXq9nXRTiH3THtK/MWf3BgGZJ9rgEOmtQ7fv46pxvsHOiXm0LXS6vZ2gpYCaDtfNGyUW0mriTSbRW3c/ch+7WQDxIoLXZzRXFbqEvkUwyiktNTLNghbpCqmfilxN17tPedtN/Pnt21urAezFMMtjKg07O3wxLe9EB7KtmA2FR18hzoQrgMFkA2caSs/8/VyvLnTIu8RtH2b8ClpTnj5klZydUzW63VZ+6IoMJ2C5jehTufpZJ+ae59gSDmC9a6mXnz37/AghoftDMX3ihUfzJt1UPtazCLVksqf7OnHUAvhYQIWJzjOuhtf92AU+Tb/YoFOKdh/XI+RmAzgWycIUb4bQAVo3+d1pKowogeG2DV4I68au0hZa6/EGtvffS/ppuVvDglUKfgb0DC6vuMXflwnGzzYFzW2Ohm/3LWmEJ1eD3xKEIT++OSNe84cgUcIa1IuFVLnvCCMkccBKfIRGNdkRIDE6T/1R+LNoHS/w/UmVXEkbL2Ds6lVCjR4vMLjgKRxeBG9yWFyY90q43JmL/YVDc1xyr81iwAvQvnUKs4f9+HQXs0SIRYhey6any9nlexF/io3SCBz6e9S1g2xUwNAi+UKGvFN1+GGU+/hDdmctpPml+g7fNZ9mAGfv4KMlw/cu1hKGt+HcuE4L/gRrX8hKkgWs5GlEMVjS9wlXbWl39o3J9i9oFUMa6ZMJsVJgvF5J+a1cjqQVqj93mAxGIXO0SLQsN5O3LCOZ5utJAyYEgLaKoyDYlSMMb+boV9k9GX8WHZ2h5a9GxNh7NC9SPBDoByJqMoNnAn937MplQ68V9K+zX8W1gJpQ=) 
[INFO] UAM (retry: 0) -> (jschl_vc param: 02dffc83af62fee9475797ae2cfbbfef) 
[INFO] UAM (retry: 0) -> (pass param: 1585084269.973-4NVLrs/Lht) 
[INFO] UAM (retry: 0) -> (jschl_answer param: 38.2423228779) 
[ERROR] Captcha (retry: 1) -> not supported! Unknown error -> Captcha page is not supported!

Scenario 2

I attempted to troubleshoot myself, making sure i had all the headers it asks for, and copied the user-agent i had working on v2, It no longer says its not supported, but it appears the clearance cookie never gets received. Once again using the CFCurlTest.php example:

Again, changing the url to:

$url = "https://torrentz2.eu/";

and using the my updated headers (see below details)

curl_setopt($ch, CURLOPT_HTTPHEADER,
    array(
        'Connection: Keep-Alive', 
        'Accept-Language: *', 
        'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:54.0) Gecko/20100101 Firefox/54.0', 
        'Accept: */*', 
        'Accept-Encoding: deflate, gzip', 
        'Upgrade-Insecure-Requests: 1'
    ));

Results of Scenario 2 This too will return NULL for $page. Occasionally i have gotten the cloudflare challange page, but mostly its NULL. This is the Verbose Error:

[ERROR] UAM (retry: 0) -> clearance cookie missing! Unknown error -> CF clearance could not be found!

See details below for Verbose Output

[INFO] UAM (retry: 0) -> applying HTTPS settings 
[INFO] UAM (retry: 0) -> waiting for 5 seconds! 
[INFO] UAM (retry: 0) -> getting form params... 
[INFO] UAM (retry: 0) -> (r param: 1c15126b1a4f21bf31cbb03224a0138d17ab8bb1-1585085734-0-Aa2fmXjZW33Mkumu6xDHTNVXrg8cqJQj1OYaFfCUV7I7O6VU3+Jm1eUCUUPqFq3yITFQN/HWFLEyzW6Mp74W9u8RigMF9AIaWGrkd3k0d+AnFtDKtu6XbT3WesJr/V7euC8yuvNFFoUjS3kNoU+wiAeiMzL17pom84IVnjGvO7Eg1bYVMSXrYCsbcLHNyTMg2rrqN4BSfOaGbdqZVn0UzaMAgAZ/8BgISoTK773bTK6t7rc2JjfAQripPnUXgIsAAEmLtpv1bau58eR2IIfKX9Wl2BNKQF/3IUFqngvZGaD9BDg0tlmwmowGYGb/YtBJmC3LOEOSOM3v9JvoMWgnZxCz7Q5LCt01KLxahKPvNXmF7BbnW/KSMU3sN6taQuPCSIbqRkzDQL421qCNJfNsaZrdi3gznhEsv84sISOozw4Grn2Qnwj1jwvhohaG9qwuJ5SPUVL4flPnhAQWjFEXckRc6mC+Wid6oz1i9RymueklY1nj7+eHyhvCt5XdTnhOPKAZv1voYu+oT+nlI6onYmMUYZAOD/cEm2hNQPWsDKXFxUbwybymoLRa4O9uf8o9uzVXWIEpcrHtsBoHjrZ956UKQ8B62YUh94ORxL+3M5TWeUbEj2eyUXZzgO7YJeyuGHQptCbmP1e+XqQ+sBBNreDk6Dca0TBStW3JYOe6otZXgH0XCWQZ589FdxJCEbh+43y1z/PymaYI7aMnof9De3g2MfM0vS6mvFg15mqOHgBWQ07CRv8p52JYQ2uwNO/XiMacJfa1pYahK3uRd9eLQ4/I6kqoZvq6X+eM6rmiM1VeE8ah4ygTdojB9p3EWoKO6Qxpq/HFJwmK1nF8wQm79DGqXBJUBWjkctZUgCQ+K7/lvvmfkYPI/ctwobKiH7ZendU8j+aqIMojxCkG/fMCdqNkxk8IeYg+hygtKciYiDkzPpRnQE6sYxOWa5J4MzBmKMPRZINIoHdEiiPKYyDC2jo0opn9E3d9yNPrm96zU6/8mopwfxWY6NY9cwdX5f52njJZyDTwfe48Ku6CvXNCwTPO6PUVIzjh/NZRhHVBR4s45QfH15P4o5HCzdXeFE8+kca49m3XPTpMcV+d16NFBMii89DLNUycH4KlWfZu1KFvsXVKbaDjVQO3bElijomcmKGvKn0wv0HsJc64FbHzsTX54W424tdQ7V3vo2k+5ep2HXL3z7OYGn3fG+AtoPtW1/7lUMFd0U5PPrE3FLMdVCRHZm3ez5V3bKlys9Wvn5KPh/a3odzSRL/WIqGzuBK1mg7j6y65XoVChFgNhp6mkD4mAHpQ2NH8HATgm+kucxDX51okTSYc0PsCWhmlL2PrnoKsn9C1Pv0lKtLa0ndskC8+7iUyNAw5TRQJKbGingMSMg8Rpe8ylYE2GV49KH31q1Y/WzzBao2y9Eedq/f8bZaJlkICrlibjJnsQNbbqDOJAQBAykRVYTJ8C9YWHcNmW2Ndyc4hMAtXVA7xL2XI5WRY86G91fpIpJsXFFaiVXWz91C2UzTo5St5KMKTdpjN8qWazzBNCilq7OeF7yWlhE1B9OfrNtzjqi0dHC6PF3Bg6hvmDHvTPgIR+M0AcO7wpa1vw4wT+wUZc+mKQRDAH/Xv+hzNawmpE5ywMwL7coHL) 
[INFO] UAM (retry: 0) -> (jschl_vc param: fef17c39adbec54945a6dcba1b85b934) 
[INFO] UAM (retry: 0) -> (pass param: 1585085738.47-iD5Lc/livn) 
[INFO] UAM (retry: 0) -> (jschl_answer param: 11.1452858565) 
[INFO] Array ( [0] => #HttpOnly_.torrentz2.eu TRUE / FALSE 1586884559 __cfduid da4ce72aadfbac86cc8af75b53ca048fc1584292559 [1] => #HttpOnly_.torrentz2.is TRUE / FALSE 1587675328 __cfduid d43232bc801bf495b5136fa5a9aae30b91585083328 ) 
[ERROR] UAM (retry: 0) -> clearance cookie missing! Unknown error -> CF clearance could not be found!

Additional information you deem important (e.g. issue happens only occasionally): This issue has been posted before, to which either it goes ignored, or someone replies "that site works for me" and the issue is ignored or assumed fixed.

I understand it may be a setup issue, rather than a bug, but i'm using the stock example, so something isn't as it should be

Version you are using (you may find this information in your composer.lock file)

Note: I've updated fully with composer and those files show as modified in September, 2019, I tested both scenarios, and got the above results. I then downloaded the latest commit zip file, with files updated today, and copied those overtop, tested both scenarios again

"version": "3.3.0"

PHP Version you are using

PHP Version 7.3.15
jaymoulin commented 4 years ago

You're using a deprecated version of cloudflare-bypass as you're using an old deprecated version (3.3.0). As @KyranRana is currently not using a stable tag release process (cf #157 ), the only possible way for you to use the actual latest with composer is to depend on jaymoulin/cloudflare-bypass instead of kyranrana/cloudflare-bypass. Another possible way is to download the actual master branch and paste all the codebase in your vendor folder. (cf https://github.com/KyranRana/cloudflare-bypass/issues/164#issuecomment-599148216)

Once again, I'm not happy with this (hopfully temporary) solution but this is the only way I found to unstuck this situation.

nitemare commented 4 years ago

You're using a deprecated version of cloudflare-bypass as you're using an old deprecated version (3.3.0). As @KyranRana is currently not using a stable tag release process (cf #157 ), the only possible way for you to use the actual latest with composer is to depend on jaymoulin/cloudflare-bypass instead of kyranrana/cloudflare-bypass. Another possible way is to download the actual master branch and paste all the codebase in your vendor folder. (cf #164 (comment))

Once again, I'm not happy with this (hopfully temporary) solution but this is the only way I found to unstuck this situation.

If you read my post, i twice stated that i downloaded the zip, and overwrote the files with the newer one, i have the same files that are found by clicking 'Download Zip' under 'Clone or Download' on this page: https://github.com/KyranRana/cloudflare-bypass

Further, i get the same results using jaymoulin/cloudflare-bypass

 
I have installed with composer, as well as downloaded the zip and copied the files over the composer-version files.
Note: I've updated fully with composer and those files show as modified in September, 2019, I tested both scenarios, and got the above results. I then downloaded the latest commit zip file, with files updated today, and copied those overtop, tested both scenarios again
jaymoulin commented 4 years ago

I read your post. I suggested this because your version indicates 3.3.0 which is not the latest version.

You can see here https://github.com/KyranRana/cloudflare-bypass/runs/531659676, here https://github.com/KyranRana/cloudflare-bypass/suites/544653472/artifacts/3335930, and here https://github.com/KyranRana/cloudflare-bypass/blob/master/src/test/FunctionalTest.php#L30 that the actual version is working for CF clearance page.

As explained in both code and your reported error, [ERROR] Captcha (retry: 1) -> not supported! Unknown error -> Captcha page is not supported!

Also, please keep in mind I'm trying to help you and I don't think attitude will cheer people to help.

momala454 commented 4 years ago

try to reorder the headers, you have

"Upgrade-Insecure-Requests: 1",

"User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36",

"Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3",

"Accept-Language: en-US,en;q=0.9"

do

"Accept-Language: en-US,en;q=0.9",
"User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36",
"Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3",
"Upgrade-Insecure-Requests: 1",
nitemare commented 4 years ago

I'm not trying to give you attitude, but i am understandably frustrated that i have manually copied all files from the zip file, and installed them in the vendor folder, making it the latest version, and yet you tell me its not? and don't tell me where a newer version would be, i don't see how i could have an 'old deprecated version'

I also have the same issues with using jaymoulin/cloudflare-bypass

@momala454 Thanks for the suggestion, i just tried that and got the same result

momala454 commented 4 years ago

test using the code from https://github.com/KyranRana/cloudflare-bypass/blob/master/src/test/FunctionalTest.php#L30

nitemare commented 4 years ago

test using the code from https://github.com/KyranRana/cloudflare-bypass/blob/master/src/test/FunctionalTest.php#L30

I've copied and pasted it, and used the below as the final code


require "{Home Path}/libraries/vendor/autoload.php";

use CloudflareBypass\CFCurlImpl;
use CloudflareBypass\Model\UAMOptions;

$url = "https://torrentz2.eu/";
        $ch = curl_init($url);
        curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
        curl_setopt($ch, CURLINFO_HEADER_OUT, true);
        curl_setopt($ch, CURLOPT_HTTPHEADER,
            array(
                'Accept-Language: *',
                'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:54.0) Gecko/20100101 Firefox/54.0',
                'Accept: */*',
                'Upgrade-Insecure-Requests: 1'
            ));

        $cfCurl = new CFCurlImpl();

    $cfOptions = new UAMOptions();

    try {
        $page = $cfCurl->exec($ch, $cfOptions);

    } catch (ErrorException $ex) {
    echo "Unknown error -> " . $ex->getMessage();
    }

Edit: forgot to mention, the above code still returns error 'Unknown error -> Captcha page is not supported!'

momala454 commented 4 years ago

if you have a captcha it means your ip is blacklisted

KyranRana commented 4 years ago

Sorry guys I am going to tag the most recent commit of this project version 3.4.0. Big thanks to @jaymoulin