berstend / puppeteer-extra

💯 Teach puppeteer new tricks through plugins.
https://extra.community
MIT License
6.39k stars 738 forks source link

[Bug] Occasional hangs when connecting to a chrome instance using the browserWSEndpoint option #680

Open Molay opened 2 years ago

Molay commented 2 years ago

Describe the bug

When I tested the browserless service, I found that using puppeteer-extra with the puppeteer-extra-plugin-stealth plugin to connect to the chrome instance through the browserWSEndpoint option and using page.goto with { waitUntil: 'networkidle2' } parameter sometimes cause a hang.

In this situation when the browserless service ends the session and closes the WebSocket, puppeteer-extra will not dispatch page error event, and the page instance remains suspended.

After testing, the browserless web terminal can normally sense the disconnected state of the WebSocket.

Maybe it's a bug similar to #676

Code Snippet

const puppeteerExtra = require('puppeteer-extra')
const StealthPlugin = require('puppeteer-extra-plugin-stealth')
puppeteerExtra.use(StealthPlugin())

;(async () => {
  const browser = await puppeteerExtra.connect({
    // Connect to browserless service
    browserWSEndpoint: `wss://xxxxxx`,
  })
  browser.on('disconnected', () => {
    // This event is indeed dispatched when the WebSocket session ends 
    console.log('browser disconnected.');
  })
  const page = await browser.newPage()
  // Notice: Since this webpage uses WebSocket to update the data, the backend data keeps flowing in
  const url = `https://finance.yahoo.com/cryptocurrencies`
  // Sometimes hangs even if the chrome instance is destroyed and the WebSocket is closed, no error event dispatched
  await page.goto(url, {
    waitUntil: 'networkidle2',
  })
  await page.waitForTimeout(1000 * 5)
  const outerHTML = await page.evaluate(() => {
    return document.documentElement.outerHTML
  })
  console.log('finished:', outerHTML)
})()

Versions

System: OS: macOS 12.4 CPU: (8) x64 Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz Memory: 171.30 MB / 16.00 GB Shell: 5.8.1 - /bin/zsh Binaries: Node: 18.0.0 - /usr/local/bin/node Yarn: 1.22.19 - ~/.yarn/bin/yarn npm: 8.6.0 - /usr/local/bin/npm npmPackages: puppeteer-core: ^15.3.2 => 15.3.2 puppeteer-extra: ^3.3.4 => 3.3.4 puppeteer-extra-plugin-stealth: ^2.11.0 => 2.11.0

berstend commented 2 years ago

Hmm, unable to reproduce this as I'm not using browserless.

Two things come to mind: