berstend / puppeteer-extra

💯 Teach puppeteer new tricks through plugins.
https://extra.community
MIT License
6.52k stars 744 forks source link

[Bug] puppeteer stealth mode is not working with browser.pages #513

Open JoeELToukhy opened 3 years ago

JoeELToukhy commented 3 years ago

Stealth mode is not working with array of pages

I am using stealth mode in chrome headful and its working but when I change to headless the stealth mode didn't work with it, I have figured the problem because I using array of pages and I have to use this in that way for other isseues with bot detection.


const puppeteer = require('puppeteer-extra')
const StealthPlugin = require('puppeteer-extra-plugin-stealth')

puppeteer.use(StealthPlugin())

;(async () => {
  const browser = await puppeteer.launch({
  headless: true,
  executablePath: '/path/to/Chrome'
  })
  const [ page ] = await browser.pages()
  await page.goto('https://bot.sannysoft.com')
  await page.waitForTimeout(5000)
  await page.screenshot({ path: 'testresult.png', fullPage: true })
  await browser.close()
})()

Versions puppeteer-extra@3.1.18 puppeteer-extra-plugin-user-preferences@2.2.12

GiveDaData commented 3 years ago

I think the default page does not have plugin enabled

let page = await browser.newPage(); let pages = await browser.pages(); const oldPage = pages[0]; oldPage.close(); I do this to have only one page, but const page = await browser.newPage(); should be fine for headless

JoeELToukhy commented 3 years ago

thats exactly my problem, I have to use the default page, because the website I am working on can detect the bot when using const page = await browser.newPage();

GiveDaData commented 3 years ago

If I was correct about the default page not applying plugin. Then, when you thought it worked for you in headfull with the default page. It may have been ran without the stealth plugin. And, in headless they got you without the stealth plugin.

When you use const page = await browser.newPage(); and stealth plugin is applied, they might also get you through other method.

edit: what do you mean by "bit detection"? first time hearing it

JoeELToukhy commented 3 years ago

What makes me say its bug that in my way it works headful, stealth plugin applied on default page only in headful not in headless.

So it was typing error i meant bot

prescience-data commented 3 years ago

The evasions are not applied when accessing the page via an array.

JoeELToukhy commented 3 years ago

The evasions are not applied when accessing the page via an array.

Is it possible to apply it on array of pages or on default page?

robintan commented 3 years ago

Since we the headless stealth is not working on default page, i.e. const page = (await browser.pages())[0], we cannot use the incognito page actually, since browser.newPage() opens up a new page in a non-incognito context

const browser = await puppeteer.launch({
  headless: true,
  args: [
    '--no-sandbox',
    '--incognito',
  ]
});