[BUG] Await page.waitForLoadState('load'); doesn't wait in headless mode

[denkorz]

Context:

• Playwright Version: 1.28.1
• Operating System: Windows 10 10.0.19044
• Node.js version: 19.0.1
• Browser: Chromium

Code Snippet

test('playwright investigation', async ({ context }) => {
    const page = await context.newPage();
    await page.goto('https://playwright.dev/');
    await expect(page).toHaveTitle(/Playwright/);
    const twitterLink = page.locator("//a[contains(@href, 'twitter')]");
    let [twitterPage] = await Promise.all([
      context.waitForEvent('page'),
      twitterLink.click({button: "middle"})
    ]);
    await twitterPage.waitForLoadState('load');    
    expect(await twitterPage.title()).toBe("Playwright (@playwrightweb) / Twitter");
  });

If this code run in debug mode or with a headed version of the browser, it works OK and the test is green (I can see that the await twitterPage.waitForLoadState('load'); line takes about 2,5 seconds). But if I run it in headless mode, the test fails on the assert. The actual twitterPage title is an empty string. And also this code runs about 5ms await twitterPage.waitForLoadState('load'); (so this waiter doesn't work). I can add explicit wait and in this case test is green in headless mode, so the problem is with the waitForLoadState method in headless mode.

[denkorz]

I think this also might be related to this issue https://github.com/microsoft/playwright/issues/12342

[aslushnikov]

@denkorz it looks like Twitter employs anti-bot detection since it doesn't want to return content to headless chrome.

The following fails as well:

import { test, expect } from '@playwright/test';

test('playwright investigation', async ({ context }) => {
  const page = await context.newPage();
  await page.goto('https://twitter.com/playwrightweb');
  await expect(page).toHaveTitle("Playwright (@playwrightweb) / Twitter");
});

Anti-bot evasion falls outside of the project, so closing this.

P.S. note the usage of the await expect(page).toHaveTitle() method to await page title.

[denkorz]

@aslushnikov

BUT, if I add explicit wait, all this logic works. Just this line await new Promise(f => setTimeout(f, 5000)); before checking the title and the test works in headless mode. I also tried to get a screenshot which has to be available in headless mode, but it's empty for some reason even for your main page.

[aslushnikov]

@denkorz twitter doesn't load for me in headless mode at all; does my snippet from https://github.com/microsoft/playwright/issues/19365#issuecomment-1344572449 work for you?

[denkorz]

@aslushnikov yes, but now also my test works fine in headless and I can see waitForLoadState takes several seconds. By the way, since this bug is floating, is there a way to get access to internal playwright logs from a test? At least I'll be able to provide some logs next time.

async waitForLoadState(state = 'load', options = {}) {
    state = verifyLoadState('state', state);
    return this._page._wrapApiCall(async () => {
      const waiter = this._setupNavigationWaiter(options);
      if (this._loadStates.has(state)) {
// -----this log------
        waiter.log(`  not waiting, "${state}" event already fired`);
      } else {
        await waiter.waitForEvent(this._eventEmitter, 'loadstate', s => {
// -----this log------
          waiter.log(`  "${s}" event fired`);
          return s === state;
        });
      }
      waiter.dispose();
    });
  }