microsoft / playwright

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
https://playwright.dev
Apache License 2.0
66.32k stars 3.63k forks source link

[Feature]: Hide `route.*` calls from test traces / reports #30970

Closed chrissantamaria closed 3 weeks ago

chrissantamaria commented 4 months ago

🚀 Feature Request

In tests, it can often be useful to use page.route to globally intercept page requests (say, blocking requests matching a certain pattern) using route.continue() or route.abort():

await page.route('**/*', (route) => {
  if (someCondition) {
    return route.abort();
  }
  return route.continue();
});

However, this results in many steps being included in test traces / reports which can make it tricky to spot other, more explicit steps from test logic.

Example trace, where the left sidebar is filled with lots of route.continue calls:

Screenshot 2024-05-22 at 7 04 54 PM

In reports, this generally isn't as significant given that adjacent calls will be grouped together:

Screenshot 2024-05-22 at 7 09 55 PM

However, if a test flips between allowing and rejecting requests it can get a bit noisy:

Screenshot 2024-05-22 at 7 03 36 PM

Ideally, it would be nice if these could be hidden. That could take the form of:

This is similar to #28482, though I see that issue was closed since there was no reproduction provided. I think test reports could be improved, but I'm particularly interested in improvements to the trace viewer where adjacent route.* calls are not grouped together.

Happy to provide a formal reproduction, but I was able to create the first two screenshots with this simple test:

import { test } from '@playwright/test';

test('example', async ({ page }) => {
  await page.route('**/*', (route) => {
    return route.continue();
  });

  await page.goto('https://playwright.dev/');
});

Example

No response

Motivation

In longer tests, there can often be hundreds of network requests which fill the UI (particularly for traces) and make it tricky to find steps related to actual test assertions. If Playwright supported hiding these (even if temporarily), it could make test reports much simpler to parse.

chrissantamaria commented 2 months ago

31726 looks really nice! Will try it out once the next alpha is published.

I wonder if there's a way for route actions to be hidden by default when opening traces. For my use case, there are many engineers interacting with traces that might not realize the setting exists - it would be nice to give them a sensible default for our project where route actions aren't relevant. Similarly, even for those who do know about the setting it's a bit tedious to have to navigate through and update on each trace load.

Might it make sense to have a default setting option defined in the Playwright config? Generated traces could honor that setting and pass it as the default value to useSetting.

Completely understand if there's hesitation in adding more config fields - just bringing up as food for thought. If we go that route, I'm also happy to help contribute.

dgozman commented 2 months ago

@chrissantamaria The setting should persist between trace viewer launches, so you will need to only toggle it once.

chrissantamaria commented 2 months ago

Oh, nice, missed that bit in useSetting. I suppose that addresses the second concern of needing toggle on each load, but folks will still need to know the setting exists. Not a huge deal though, and maybe not worth the overhead of extending the Playwright config schema.

Thanks for working on this!

dgozman commented 2 months ago

Trace Viewer / UI mode change has landed, moving this issue to the html report.

kvnschlndt commented 1 month ago

@pavelfeldman when is this feature also expected to be available for html report? We have a highly noisy and hardly readable test report because of a lot of modifcation to requests/responses.

kvnschlndt commented 1 month ago

@pavelfeldman when is this feature also expected to be available for html report? We have a highly noisy and hardly readable test report because of a lot of modifcation to requests/responses.

@pavelfeldman ?

samSam-0 commented 1 month ago

is this gonna be available for html report and allure report? @dgozman @pavelfeldman @mxschmitt

baev commented 1 month ago

Wouldn't it be best to disable reporting for all route steps by default (or via the configuration option) to fix the problem for all reporters (including 3rd party)?

Is there any use for route events in reporters?

dgozman commented 3 weeks ago

Should be addressed by #32723.