search

berstend / puppeteer-extra

💯 Teach puppeteer new tricks through plugins.
https://extra.community
MIT License
6.38k stars 738 forks source link

puppeteer-extra-plugin-recaptcha doesn't work on crunchyroll or Tidal #247

Closed sgau77 closed 4 years ago

sgau77 commented 4 years ago

I can't get the recaptcha plug in to work on crunchyroll or Tidal. Here's my example code for Tidal. It's showing that there's no captcha on the page when it is clearly there in an iframe.

const puppeteer = require('puppeteer-extra')

// add recaptcha plugin and provide it your 2captcha token
// 2captcha is the builtin solution provider but others work as well.
const RecaptchaPlugin = require('puppeteer-extra-plugin-recaptcha')
puppeteer.use(
  RecaptchaPlugin({
    provider: { id: '2captcha', token: 'XXXXXXX' },
    visualFeedback: true, // colorize reCAPTCHAs (violet = detected, green = solved)
  })
)

puppeteer.launch({ headless: false }).then(async (browser) => {
  const page = await browser.newPage()
  await page.goto('https://my.tidal.com/login')
  await page.waitFor('[value="Log in"]')
  // const loginBtn = '[value="Log in"]'
  // await page.evaluate((loginBtn) => document.querySelector(loginBtn).click(), loginBtn)
  await page.click('[value="Log in"]')
  await page.waitFor('input[id="email"]')
  await page.waitFor(5000)
  await page.type('input[id="email"]', 'panda@yahoo.com', { delay: 50 })
  await page.click('[id="recap-invisible"]')
  await page.waitFor(10000)
  await page.screenshot({ path: 'tidal-input-username.png' })
  const { captchas, solutions, solved, error } = await page.solveRecaptchas()
  console.log('captchas', captchas)
  console.log('solutions', solutions)
  console.log('solved', solved)
  console.log('error', error)
  await page.waitFor('input[id="password"]')
  await page.waitFor(40000)
  await page.screenshot({ path: 'tidal-solved-captcha.png' })
  await browser.close()
})
berstend commented 4 years ago

Have you checked if the reCAPTCHA is in an iframe?

      for (const frame of page.mainFrame().childFrames()) {
        await frame.solveRecaptchas()
      }

Also please provide debug output (as mentioned in the readme) to make it easier for us to see what's going on from the perspective of the reCAPTCHA plugin. :-)

andrew-healey commented 4 years ago

I don't think the reCAPTCHA is inaccessible in the main browser context; ___grecaptcha_cfg is defined and contains the correct clients property (at least in the test I ran).

sgau77 commented 4 years ago

I tried what you suggested and I'm still getting the same issue. Here's the terminal logs. The wait for selector failed because the captcha was not solved and still on the page. From the terminal response, it seems the plug in was unable to detect or solve the captcha. The Tidal login page does use the vue framework so I don't know that itself is causing an issue.

Here's my code

const puppeteer = require('puppeteer-extra')
const RecaptchaPlugin = require('puppeteer-extra-plugin-recaptcha')
puppeteer.use(
  RecaptchaPlugin({
    provider: { id: '2captcha', token: 'xxxxx' },
    visualFeedback: true, // colorize reCAPTCHAs (violet = detected, green = solved)
  })
)

puppeteer.launch({ headless: false }).then(async (browser) => {
  const page = await browser.newPage()
  await page.goto('https://my.tidal.com/login')
  await page.waitFor('[value="Log in"]')
  // const loginBtn = '[value="Log in"]'
  // await page.evaluate((loginBtn) => document.querySelector(loginBtn).click(), loginBtn)
  await page.click('[value="Log in"]')
  await page.waitFor('input[id="email"]')
  await page.waitFor(5000)
  await page.type('input[id="email"]', 'panda@yahoo.com', { delay: 50 })
  await page.click('[id="recap-invisible"]')
  await page.waitFor(10000)
  await page.screenshot({ path: 'tidal-input-username.png' })
  for (const frame of page.mainFrame().childFrames()) {
    const { captchas, solutions, solved, error } = await frame.solveRecaptchas()
    console.log('captchas', captchas)
    console.log('solutions', solutions)
    console.log('solved', solved)
    console.log('error', error)
  }
  await page.waitFor('input[id="password"]')
  await page.waitFor(40000)
  await page.screenshot({ path: 'tidal-solved-captcha.png' })
  await browser.close()
})

Debug log

^Cq2-MBP:recaptcha-test sgau$ DEBUG=puppeteer-extra,puppeteer-extra-plugin:* node tidal
  puppeteer-extra-plugin:base:recaptcha Initialized. +0ms
  puppeteer-extra-plugin:recaptcha Initialized { visualFeedback: true,
  throwOnError: false,
  provider:
   { id: '2captcha', token: 'xxxx' } } +0ms
  puppeteer-extra plugin registered recaptcha +0ms
  puppeteer-extra no dependencies are missing +1ms
  puppeteer-extra orderPlugins:before [ 'recaptcha' ] +1ms
  puppeteer-extra orderPlugins:after [ 'recaptcha' ] +0ms
  puppeteer-extra-plugin:recaptcha onPageCreated about:blank +0ms
  puppeteer-extra-plugin:recaptcha solveRecaptchas +0ms
  puppeteer-extra-plugin:recaptcha findRecaptchas +0ms
  puppeteer-extra-plugin:recaptcha hasRecaptchaScriptTag false +0ms
  puppeteer-extra-plugin:recaptcha _generateContentScript findRecaptchas undefined +0ms
  puppeteer-extra-plugin:recaptcha findRecaptchas { captchas: [], error: null } +0ms
  puppeteer-extra-plugin:recaptcha solveRecaptchas { captchas: [], solutions: [], solved: [], error: null } +0ms
captchas []
solutions []
solved []
error null
  puppeteer-extra-plugin:recaptcha solveRecaptchas +0ms
  puppeteer-extra-plugin:recaptcha findRecaptchas +0ms
  puppeteer-extra-plugin:recaptcha hasRecaptchaScriptTag false +0ms
  puppeteer-extra-plugin:recaptcha _generateContentScript findRecaptchas undefined +0ms
  puppeteer-extra-plugin:recaptcha findRecaptchas { captchas: [], error: null } +0ms
  puppeteer-extra-plugin:recaptcha solveRecaptchas { captchas: [], solutions: [], solved: [], error: null } +0ms
captchas []
solutions []
solved []
error null
(node:27744) UnhandledPromiseRejectionWarning: TimeoutError: waiting for selector "input[id="password"]" failed: timeout 30000ms exceeded
    at new WaitTask (/Users/sgau/node_modules/puppeteer/lib/cjs/common/DOMWorld.js:417:34)
    at DOMWorld._waitForSelectorOrXPath (/Users/sgau/node_modules/puppeteer/lib/cjs/common/DOMWorld.js:342:26)
    at DOMWorld.waitForSelector (/Users/sgau/node_modules/puppeteer/lib/cjs/common/DOMWorld.js:325:21)
    at Frame.waitForSelector (/Users/sgau/node_modules/puppeteer/lib/cjs/common/FrameManager.js:385:51)
    at Frame.<anonymous> (/Users/sgau/node_modules/puppeteer/lib/cjs/common/helper.js:110:27)
    at Frame.waitFor (/Users/sgau/node_modules/puppeteer/lib/cjs/common/FrameManager.js:376:25)
    at Page.waitFor (/Users/sgau/node_modules/puppeteer/lib/cjs/common/Page.js:1068:33)
    at puppeteer.launch.then (/Users/sgau/Desktop/puppeteer-scripts/recaptcha-test/tidal.js:33:14)
    at process._tickCallback (internal/process/next_tick.js:68:7)
(node:27744) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:27744) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
sgau77 commented 4 years ago

I don't think the reCAPTCHA is inaccessible in the main browser context; ___grecaptcha_cfg is defined and contains the correct clients property (at least in the test I ran).

@Sesamestrong , were you able to solve the captcha on Tidal? If so, do you mind sharing your code?

berstend commented 4 years ago

Are you sure a captcha is being shown during the script run? Can you run the script again and make a screenshot as well? The existence of a recaptcha script doesn't mean a captcha is being shown.

sgau77 commented 4 years ago

image

sgau77 commented 4 years ago

Here's the screenshot prior to the captcha via automation. image

andrew-healey commented 4 years ago

I don't think the reCAPTCHA is inaccessible in the main browser context; ___grecaptcha_cfg is defined and contains the correct clients property (at least in the test I ran).

@Sesamestrong , were you able to solve the captcha on Tidal? If so, do you mind sharing your code?

I was not able to solve it; I'm just mentioning that the basic idea of inspecting ___grecaptcha_cfg is not flawed; something else in puppeteer-extra-plugin-recaptcha likely is.

sgau77 commented 4 years ago

Any update on this issue?

berstend commented 4 years ago

It could be possible this has been fixed with #272 (puppeteer-extra-plugin-recaptcha@3.1.15) - would be nice if someone can confirm :)

psalkowski commented 4 years ago

I'm using webpack with latest 3.1.15 version of recaptcha plugin. Unfortunately I faced the same problem. My piece of code:

            const {solved, solutions, captchas, error} = await page.solveRecaptchas();

            if (solved.length || solutions.length || captchas.length || error) {
                console.log('Recaptcha:', {solved, solutions, captchas, error});
                await page.waitForNavigation();
            }

Error: TimeoutError: waiting for function failed: timeout 10000ms exceeded.

The error is not always throwed. Debug shows:

$ DEBUG=puppeteer-extra,puppeteer-extra-plugin:* node ./dist/main.js
  puppeteer-extra-plugin:base:recaptcha Initialized. +0ms
  puppeteer-extra-plugin:recaptcha Initialized {
  visualFeedback: true,
  throwOnError: false,
  provider: { id: '2captcha', token: 'xxxx' }
} +0ms
  puppeteer-extra plugin registered recaptcha +0ms
  puppeteer-extra no dependencies are missing +0ms
  puppeteer-extra orderPlugins:before [ 'recaptcha' ] +0ms
  puppeteer-extra orderPlugins:after [ 'recaptcha' ] +0ms
  puppeteer-extra-plugin:recaptcha solveRecaptchas +0ms
  puppeteer-extra-plugin:recaptcha findRecaptchas +0ms
  puppeteer-extra-plugin:recaptcha hasRecaptchaScriptTag true +0ms
  puppeteer-extra-plugin:recaptcha waitForRecaptchaClient - start 2020-08-13T21:50:01.848Z +0ms
  puppeteer-extra-plugin:recaptcha solveRecaptchas {
  captchas: [],
  solutions: [],
  solved: [],
  error: 'TimeoutError: waiting for function failed: timeout 10000ms exceeded'
} +0ms

What I have found is, the error is thrown when I'm on login page. When I log in, and have sessions stored, then re-launch browser directly on user panel, the error is gone.

berstend commented 4 years ago

This might be fixed in puppeteer-extra-plugin-recaptcha@3.1.18 (see #300, #304).

Closing for now, please re-open if the above didn't fix the issue.