[BUG] browserContext.newPage() sometimes hangs indefinitely

[ProQuestionAsker]

Context:

• Playwright Version: playwright-firefox 1.3.0
• Operating System: Mac
• Node.js version: 10.15.3
• Browser: Firefox
• Extra: Running in Heroku and using Heroku Playwright Buildpack

Code Snippet I'm running a script in Heroku that loops over a number of urls and takes screenshots. Every once in a while (maybe 40% of the time?) the loop will indefinitely hang. I've finally tracked it down to the newPage function.

A simplified version of my code is something like this:

const { firefox } = require('playwright-firefox');

async function takeScreenshots(page, address){
    // when new page hangs, this function is never reached
    await page.setDefaultTimeout(20000)
    await page.setViewportSize({ width: 640, height: 480})
    await page.goto(address)
    await page.screenshot()
}

async function findURL(address){
    const browser = await firefox.launch({headless: true,  timeout: 5000, args: ['--no-sandbox']})
        .catch(e => console.error(`error launching browser: ${e}`))

    const context = await browser.newContext()
        .catch(e => console.error(`Error launching new context: ${e}`))

    const page = await context.newPage()
        .catch(e => console.error(`error launching new page: ${e}`))

     await takeScreenshots(page, address).catch(error => console.error(`Error taking screenshots: ${error}`))
     await browser.close()
}

(async function loopThroughURLs(){
    const urls = [url1, url2, url3]

    for (const address of urls){
        await findURL(address).catch(error => console.error(`Error finding url: ${error}`))
    }
})().catch(error => console.error(`Error looping through urls: ${error}`))

Describe the bug

I've tried running the script with process.env.DEBUG='pw:api'. When combined with lots of console logs, I've found that typically when await context.newPage() runs, this is printed to the terminal:

2020-09-21T19:04:48.582Z pw:api   navigated to "about:blank" []
2020-09-21T19:04:48.594Z pw:api   navigated to "about:blank" []

and my script continues as expected.

But sometimes it never prints, never throws an error, and never times out. To be clear, the browser and context both seem to have successfully launched, but the new page doesn't. Sometimes this happens after 10 loops through my script, other times it happens after 1 or 2. Has anyone else stumbled upon something like this?

Many thanks for the amazing work!

Edit to add: I just tried running again with process.env.DEBUG='pw:browser' and when it hangs, I get these warnings after the context has launched:

2020-09-21T20:23:38.292Z pw:browser [Parent 1729, Gecko_IOThread] WARNING: Failed to launch tab subprocess: file /mnt/cron/playwright/browser_patches/firefox/checkout/ipc/glue/GeckoChildProcessHost.cpp, line 745 []
2020-09-21T20:23:38.293Z pw:browser [Parent 1729, Gecko_IOThread] WARNING: Failed to launch tab subprocess: file /mnt/cron/playwright/browser_patches/firefox/checkout/ipc/glue/GeckoChildProcessHost.cpp, line 745 []
2020-09-21T20:23:38.293Z pw:browser JavaScript error: chrome://global/content/elements/browser-custom-element.js, line 1113: NS_ERROR_UNEXPECTED: Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIScriptSecurityManager.getLoadContextContentPrincipal] []
2020-09-21T20:23:38.294Z pw:browser JavaScript error: chrome://browser/content/tabbrowser.js, line 432: TypeError: browser.webProgress is null []
2020-09-21T20:23:38.327Z pw:browser IPDL protocol Error: Received an invalid file descriptor []
2020-09-21T20:26:38.714Z pw:browser console.log: *** PurgeTrackerService:: "returning early, etpActive: false, purgeEnabled: true" []

[pavelfeldman]

You should move const browser = await firefox.launch out of the loop - that will speed up your script 1000 times. Also pass default viewport size into the newContext instead of setting it all the time.

[ProQuestionAsker]

Good call! With those changes and a page.on('pageerror'...) check, it seems to be working. I'll run it a few more times tomorrow and see if it pops up again. Thanks!

[dgozman]

Closing as this seems to be resolved. Please comment/reopen if the issue persists.

[jgehrcke]

You should move const browser = await firefox.launch out of the loop - that will speed up your script 1000 times. Also pass default viewport size into the newContext instead of setting it all the time.

This was nice general advice about the code shown. But unless I am missing something this issue doesn't quite discuss the expected scenarios when await context.newPage() is known to not return for a long time. We're affected by that right now, and I'm starting to debug this. Was hoping to get some more specific advice in here :).