search

codemanki / cloudscraper

--DEPRECATED -- 🛑 🛑 Node.js library to bypass cloudflare's anti-ddos page
MIT License
602 stars 139 forks source link

TypeError: Challenge evaluation failed: can't redefine non-configurable property "document" #309

Closed vvpgrp closed 4 years ago

vvpgrp commented 4 years ago

Hi @codemanki and team,

I am using cloudscraper@4.5.0 and node@10.15.1. Cloudscraper works fine when run in node. Code: 

require('cloudscraper').get('https://sochain.com/api/v2/address/btctest/2N7GbVfHioqGTHEAwftrqSxXf4mk919xySY', (e, _, r) => console.log(r));

Using Webpack, I create one js file (named as index.js) from the cloudcraper module and run it in Firefox 71.0. Code (look here: Tests.zip): 

<!DOCTYPE html>
<html style="height: auto; min-height: 100%;" lang="en-US">
 <head>
  <meta http-equiv="content-type" content="text/html; charset=UTF-8">
  <meta charset="UTF-8">
  <meta content="width=device-width, initial-scale=1, maximum-scale=1, user-scalable=no" name="viewport">
  <title>Tests cloudscraper</title>
  <script src="Tests_files/index.js"></script>
 </head>
 <body>            
   <p><button id="btnCheckMySochain" onclick="checkMySochain()">This will test cloudscraper when clicked.</button></p>      
   <script>
      function checkMySochain() {   
    var cors = 'https://cors-anywhere.herokuapp.com/';
    var options = {
       uri:cors+'https://www.sochain.com/api/v2/address/btctest/2N7GbVfHioqGTHEAwftrqSxXf4mk919xySY',       
       // uri:cors+'https://www.sochain.com/api/v2/address/btctest/tb1qhdx9nylta364u7tkc4qdtyjsekqtw8ml2p29e8',
    };

    mySochain.cloudscraper.debug = true;    

    document.getElementById("btnCheckMySochain").disabled = true;           

    var msg = 'see the results in the web console! (for Firefox: Ctrl+Shift+I)';
    mySochain.cloudscraper.get(options)
       .then(function (htmlString) {
        console.log(JSON.parse(htmlString));
        document.getElementById("btnCheckMySochain").disabled = false;          
        alert('SUCCESS: '+msg);
       })
       .catch(function (err) {
        console.log(err);
        document.getElementById("btnCheckMySochain").disabled = false;          
        alert('ERROR: '+msg);
       });
    }
    </script>
   </body>
</html>

This code works fine for a period of time from approximately 8 GMT+2 to 14 GMT+2. Other times, I get an error:

name: ParserError
message: 
### Cloudflare may have changed their technique, or there may be a bug.
### Bug Reports: https://github.com/codemanki/cloudscraper/issues
### Check the detailed exception message that follows for the cause.

TypeError: Challenge evaluation failed: can't redefine non-configurable property "document"
constructor: function n(e,t,a){if(e instanceof s)return e;Object.defineProperties(this,h),s.apply(this,arguments),this.name=i,"function"==typeof r&&r(this),Error.captureStackTrace&&Error.captureStackTrace(this,n)}
errorType: 3

Any help would be greatly appreciated. Thanks in advance. Vasyl

Please attempt to answer the following questions before submitting a new issue:

Debug information ```js REQUEST { requester: { [Function: n] get: [Function], head: [Function], options: [Function], post: [Function], put: [Function], patch: [Function], del: [Function], delete: [Function], jar: [Function], cookie: [Function], defaults: [Function], forever: [Function], Request: { [Function: L] super_: [Object], debug: true, defaultProxyHeaderWhiteList: [Object], defaultProxyHeaderExclusiveList: [Object] }, initParams: [Function: p], debug: [Getter/Setter], bindCLS: [Function] }, jar: { _jar: { enableLooseMode: true, store: { idx: {} } } }, headers: { Host: {}, Connection: 'keep-alive', 'Upgrade-Insecure-Requests': '1', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.2; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.40 Safari/537.36', Accept: 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3', 'Accept-Language': 'en-US,en;q=0.9', 'Accept-Encoding': 'gzip, deflate, br' }, cloudflareMaxTimeout: 30000, followAllRedirects: true, challengesToSolve: 3, decodeEmails: false, gzip: true, agentOptions: { ciphers: 'undefined:!ECDHE+SHA:!AES128-SHA' }, uri: 'https://cors-anywhere.herokuapp.com/https://www.sochain.com/api/v2/address/btctest/2N7GbVfHioqGTHEAwftrqSxXf4mk919xySY', method: 'GET', realEncoding: 'utf8', encoding: null, callback: [Function], transform: undefined, simple: true, resolveWithFullResponse: false, transform2xxOnly: false } ----------------- REQUEST onRequestResponse https://cors-anywhere.herokuapp.com/https://www.sochain.com/api/v2/address/btctest/2N7GbVfHioqGTHEAwftrqSxXf4mk919xySY 503 { 'access-control-allow-origin': '*', 'cache-control': 'no-cache', 'cf-ray': '54ab6c407a3ac1bb-IAD', connection: 'keep-alive', 'content-type': 'text/html; charset=UTF-8', date: 'Wed, 25 Dec 2019 14:14:42 GMT', 'expect-ct': 'max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"', server: 'cloudflare', 'transfer-encoding': 'chunked', vary: 'Accept-Encoding', 'x-final-url': 'https://www.sochain.com/api/v2/address/btctest/2N7GbVfHioqGTHEAwftrqSxXf4mk919xySY', 'x-frame-options': 'SAMEORIGIN' } ----------------- REQUEST response end https://cors-anywhere.herokuapp.com/https://www.sochain.com/api/v2/address/btctest/2N7GbVfHioqGTHEAwftrqSxXf4mk919xySY 200 { 'access-control-allow-headers': 'Origin,Accept,Content-Type,X-Requested-With,X-CSRF-Token', 'access-control-allow-methods': 'GET,POST', 'access-control-allow-origin': '*', 'cache-control': 'no-cache, no-store, max-age=0, must-revalidate', 'cf-cache-status': 'DYNAMIC', 'cf-ray': '54a9eebcecebcf04-IAD', connection: 'keep-alive', 'content-type': 'application/json; charset=utf-8', date: 'Wed, 25 Dec 2019 09:54:15 GMT', 'expect-ct': 'max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"', expires: 'Fri, 01 Jan 1990 00:00:00 GMT', pragma: 'no-cache', server: 'cloudflare', status: '200 OK', 'strict-transport-security': 'max-age=31536000;includeSubDomains', 'transfer-encoding': 'chunked', 'x-content-type-options': 'nosniff', 'x-download-options': 'noopen', 'x-final-url': 'https://www.sochain.com/api/v2/address/btctest/2N7GbVfHioqGTHEAwftrqSxXf4mk919xySY', 'x-frame-options': 'SAMEORIGIN', 'x-request-id': 'cffe5b3f-2603-4042-86b4-58c4761efd97', 'x-runtime': '0.066465', 'x-xss-protection': '1; mode=block' } ```
codemanki commented 4 years ago

Hi @vvpgrp. Very interesting. Why would it work only in certain periods of time? Most probably this happens because of this code - https://github.com/codemanki/cloudscraper/blob/master/lib/sandbox.js#L18

You could try putting something like:


    const cache = Object.create(null);
    const keys = [];
    const { body, href } = global;

    if(typeof document !== 'undefined') {
        return;
    }

    Object.defineProperties(global, {
      document: {
....
vvpgrp commented 4 years ago

@Codemanki thank you very much for your reply and help! Now cloudscraper is working fine. A little later, when the error reappears, I will try your advice.

codemanki commented 4 years ago

@vvpgrp my pleasure :) Please let me know if that fix helps or not. If it does, I might add it to the source code

vvpgrp commented 4 years ago

@vvpgrp my pleasure :) Please let me know if that fix helps or not. If it does, I might add it to the source code

Today cloudscraper worked fine at 8 GMT+2, but after 14:36 GMT+2 an error message appeared:

ParserError: 
### Cloudflare may have changed their technique, or there may be a bug. 
### Bug Reports: https://github.com/codemanki/cloudscraper/issues 
### Check the detailed exception message that follows for the cause. 
TypeError: Challenge evaluation failed: a is null
vvpgrp commented 4 years ago

Dear @codemanki and team, Is there a chance to solve this problem?

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.