[BUG] [firefox] `window.performance` entry is incorrect for resources when using playwright's network APIs

[nikitaeverywhere]

TL;DR when using Playwright's network API to handle requests and simulate network response delay, for instance,

  await page.route('https://website.com/**', async (route) => {
    await delay(); // Resolves in 1 second
    await route.fulfill({ contentType: 'text/plain', body: '' });
  });

Browser's performance API entry of this resource is incorrect (no delay is present):

System info

• Playwright Version: 1.32.0
• Operating System: macOS Ventura 13.2.1
• Browser: latest auto-downloaded WebKit
• Other info: node v19.6.0

Source code

For demonstration, test/dummy.test.ts:

import { test } from '@playwright/test';

const RESOURCE = 'https://dummy-resource.com/test';
const delay = async (ms = 1000) =>
  await new Promise((resolve) => setTimeout(resolve, ms));

test('dummy', async ({ page }) => {
  await page.route(RESOURCE, async (route) => {
    await delay();
    await route.fulfill({ contentType: 'text/plain', body: '' });
  });
  await page.on('console', (msg) => console.log(msg));
  await page.goto('.');

  await page.evaluate(
    async ([RESOURCE]) => {
      await (await fetch(RESOURCE)).text();

      const performanceEntry = performance.getEntriesByName(RESOURCE)[0];

      // Visualize performance entry
      console.log(`\n${performanceEntry}:`);
      for (const prop in performanceEntry) {
        if (typeof performanceEntry[prop] === 'number') {
          console.log(`${prop} = ${performanceEntry[prop]}`);
        }
      }
    },
    [RESOURCE]
  );

  await delay(500);
  console.log('\n');
});

playwright.config.ts

```typescript import { defineConfig, devices } from '@playwright/test'; export default defineConfig({ testDir: 'test', fullyParallel: false, retries: 0, workers: 1, use: { baseURL: 'http://x.com', trace: 'on-first-retry', }, projects: [ { name: 'chromium', use: { ...devices['Desktop Chrome'] }, }, { name: 'firefox', use: { ...devices['Desktop Firefox'] }, }, { name: 'webkit', use: { ...devices['Desktop Safari'] }, }, ], }); ```

Run the test:

npx playwright install
npx playwright test

Expected

It is expected that performance api entry will have duration > 1000 in every browser.

Actual

Duration in Firefox is always 0.

THE GOAL

Being able to simulate network latency in all browsers, including correct performance API entries.

[nikitaeverywhere]

@mxschmitt please also consider reopening https://github.com/microsoft/playwright/issues/22332 as even despite the workaround exists, it is clearly a Playwright bug which causes no "error" neither "load" event being fired on HTML elements (abnormal behavior).

[aslushnikov]

@nikitaeverywhere the reason for this behavior is that our Firefox network instrumentation is conceptually similar to Service Worker, thus the behavior. This is intended outcome.

Overall, I'd recommend against using Playwright request interception as a network shaping solution - it wouldn't cover all the usecases, e.g. it wouldn't throttle WebSockets.

To throttle "for real", we'd recommend user-land solutions:

• Charles Proxy is quite popular and works across OS'es.
• On Linux, network shaping could be configured per network interface with tools like tc. wondershaper is a convenience wrapper atop of tc and might be a good starting point.
• On Mac, there's Network Link Conditioner.

Hope this helps!

[nikitaeverywhere]

@aslushnikov thanks for suggestions, but neither Charles Proxy nor tc nor Network Link Conditioner will allow me to respond from the test's context (with route.fulfill).

TL;DR the window.performance API's duration in FireFox is always 0 no matter what, while it's fine in other browsers. You can't describe this as an "intended outcome", it's a bug.

You should better qualify this as a fix for Playwright, which in the worth case is about patching PerformanceEntry (using Proxies) in FireFox and return the correct computed duration based on internal request measures (which you appear to use at least for debug/verbose mode), avoiding Service Worker instrumentation if it fails to comply with standard APIs. I saw you can easily implement such fixes. CC @mxschmitt

P.S. Might be a better approach


[aslushnikov]

@aslushnikov thanks for suggestions, but neither Charles Proxy nor tc nor Network Link Conditioner will allow me to respond from the test's context (with route.fulfill).

@nikitaeverywhere that's a fair point. But again, we give no guarantees when request interception is used to shape network traffic, and recommend against it.

If you want to shape network traffic and fulfill responses, you'll be better off with a manually controlled proxy. Would this work for you?

[nikitaeverywhere]

@aslushnikov

If you want to shape network traffic and fulfill responses, you'll be better off with a manually controlled proxy. Would this work for you?

I have around 100 tests (300 if we take 3 browsers) for a javascript library. Just a few of them do perform network latency checks, which unfortunately fail in Firefox because of the mentioned bug. I do use Playwright's network controls heavily to emulate server responses in all tests.

Answering your question, yes, I could have made a separate proxy just for these few failing tests. But this would look as an ugly temporary fix because giving up on Playwright's network controls and migrating all other tests to use an outside proxy would be very time consuming and sometimes an overengineered approach.

Temporarily, I made that mentioned JavaScript proxy object to "fake" the resource timing in the test specifically for Firefox, so that it passes. But I hope this approach could be integrated into Playwright, fixing the resource timing API bug.

[aslushnikov]

But this would look as an ugly temporary fix because giving up on Playwright's network controls and migrating all other tests to use an outside proxy would be very time consuming and sometimes an overengineered approach.

@nikitaeverywhere Well, fair enough! Let me reopen this and mark as P3 to gauge community interest. We'll prioritize accordingly.

[nikitaeverywhere]

@aslushnikov thank you. Let's see.

[nikitaeverywhere]

I've got some people asking me about the resolution of this problem (among 4 upvotes), so I'll share my "ugly workaround".

/**
 * Fixes performance API when using custom resource handling.
 *
 * @see https://github.com/microsoft/playwright/issues/22355
 */
const fixFirefoxResourceTimings = async (page: Page) => {
  await page.addInitScript(() => {
    window.__FF_HARDCODED_REQUEST_DURATION =
      window.__FF_HARDCODED_REQUEST_DURATION || [];
    const PerformanceEntry = Object.getPrototypeOf(
      Object.getPrototypeOf(Object.getPrototypeOf(performance.getEntries()[0]))
    ); // === PerformanceEntry
    const originalDescriptor = Object.getOwnPropertyDescriptor(
      PerformanceEntry,
      'duration'
    );
    Object.defineProperty(PerformanceEntry, 'duration', {
      enumerable: true,
      configurable: true,
      get: function () {
        const hardcodedEntry = window.__FF_HARDCODED_REQUEST_DURATION.find(
          (o) => o.name === this.name
        );
        if (hardcodedEntry) {
          console.log(
            `  💬 Returning hardcoded performance entry with duration=${hardcodedEntry.duration} for ${this.name}`
          );
        }
        const originalValue = originalDescriptor?.get?.call(this);
        return hardcodedEntry
          ? hardcodedEntry.duration
          : typeof originalValue !== 'undefined'
          ? originalValue
          : 42;
      },
    });
  });
};

This just redefines the performance.duration getter and lets you add window.__FF_HARDCODED_REQUEST_DURATION in a specific test which was a deal in my case.