ParseError / Challenge form is missing secret input

[drdokk]

Please attempt to answer the following questions before submitting a new issue:

• What version of Cloudscraper are you using?
  • 4.3.0
• What version of Node.js are you using? (Please share the process information)
  • v12.13.0
• When did the problem start occurring?
  • Today (afternoon)
• How often does the problem occur?
  • Always (every/any request)

[codemanki]

Please update to 4.4.0

[drdokk]

Hi @codemanki thanks for the update but i'm still getting same issue after updating to 4.4.0

Kind regards,

[codemanki]

@drdokk could you please share your code and url you are trying to scrape? If you don't want to do it publicly then just send it to me@codemanki.com

[drdokk]

Hi @codemanki sorry for late response, here is a code snippet

const tr = require('tor-request');

function onCaptcha (options, response, body) {
    const captcha = response.captcha;
    console.log(captcha);
    //captcha solve code here...
  }

const cloudscraper = require('cloudscraper').defaults({
    requester: tr.request,
    gzip: false,
    cloudflareTimeout: 5000,
    cloudflareMaxTimeout: 30000,
    followAllRedirects: true,
    onCaptcha
});

function getRequest() {
    const newjar = tr.request.jar();
    const request = cloudscraper.defaults({jar: newjar});

    return request;
}

getRequest().get('https://www.cloudflare.com/', {}, function(err, res, body) {
    console.log(err, body);
});

Output:

ParserError:
### Cloudflare may have changed their technique, or there may be a bug.
### Bug Reports: https://github.com/codemanki/cloudscraper/issues
### Check the detailed exception message that follows for the cause.

Challenge form is missing secret input
    at onCaptcha (C:\projectdir\node_modules\cloudscraper\index.js:470:21)
    at onCloudflareResponse (C:\projectdir\node_modules\cloudscraper\index.js:226:14)
    at onRequestResponse (C:\projectdir\node_modules\cloudscraper\index.js:205:5)
    at Request.<anonymous> (C:\projectdir\node_modules\cloudscraper\index.js:149:7)
    at Object.onceWrapper (events.js:300:26)
    at Request.emit (events.js:210:5)
    at Request.<anonymous> (C:\projectdir\node_modules\request\request.js:1161:10)
    at Request.emit (events.js:210:5)
    at IncomingMessage.<anonymous> (C:\projectdir\node_modules\request\request.js:1083:12)
    at Object.onceWrapper (events.js:299:28)
    at IncomingMessage.emit (events.js:215:7)
    at endReadableNT (_stream_readable.js:1183:12)
    at processTicksAndRejections (internal/process/task_queues.js:80:21) {
  name: 'ParserError',
  message: '\r\n' +
    '### Cloudflare may have changed their technique, or there may be a bug.\r\n' +
    '### Bug Reports: https://github.com/codemanki/cloudscraper/issues\r\n' +
    '### Check the detailed exception message that follows for the cause.\r\n' +
    '\r\n' +
    'Challenge form is missing secret input'
} undefined

[drdokk]

index.js:467

  // Sanity check
  if (!payload.s) {
    cause = 'Challenge form is missing secret input';
    return callback(new ParserError(cause, options, response));
  }

this code is no more valid for new recaptcha as there is no "s" field in new form, can anyone explain why is this check still there ? Thanks

Kind regards,

[codemanki]

@drdokk could you share the full html of the page with recaptcha? I will adjust cloudscraper to properly handle it

[drdokk]

Here is the full html response : https://pastebin.com/FuEEfj85

[drdokk]

This form may be only for tor traffic I don't really know, can't get to see captcha page on my real IP

[codemanki]

4.5.0 was just published