microsoft / playwright

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
https://playwright.dev
Apache License 2.0
65.58k stars 3.57k forks source link

[Feature]: Bring back old trace viewer features #30915

Closed PsiKai closed 3 months ago

PsiKai commented 3 months ago

🚀 Feature Request

On older versions of playwright (pre 1.37), the trace viewer had a feature that made debugging failed tests a lot easier.

Example

Grouped network/console activity

There is a little X that appears next to a test step where some error occurred. With all the network and console activity grouped into one long timeline, it's near impossible to associate the error in the step to this output.

When an application has dozens of requests to similar endpoints, and dozens of console output messages, with intermingling of known errors, deprecation warnings, etc., reporting on errors caused by new bad code, or flaky tests are increasingly difficult.

Screenshot 2024-05-20 at 1 13 56 PM

Motivation

I have triaged countless failed tests against a complex codebase with this feature. With it gone in the latest versions of playwright, I fear that debugging will become even more difficult. I don't think it's a huge ask to bring back what used to exist. Please bring it back.

ben-redd commented 3 months ago

I definitely agree that this needs to go back to how it was. Having to scroll through thousands of logs in the console in order to find an error in the trace viewer is taking a massive step backwards. I wonder if this could even be considered a bug because I find it hard to believe that this would be intentional.

View of what my trace viewer looks like now, with just a massive list of logs for both the console and network tabs

Screenshot 2024-05-20 at 2 50 16 PM
pavelfeldman commented 3 months ago

You should double-click the step. It'll select the time region and narrow down the console and network lists .

ben-redd commented 3 months ago

@pavelfeldman, was this ever listed in the release notes? I never saw it when upgrading, and in all honesty I feel like it's a much less user friendly experience. I'm glad we still have the option to view the console and network tab for specific actions, but double clicking to see the logs and then having to click "show all" to go back to the list of actions just seems like a lot of extra steps compared to how it used to be.

pavelfeldman commented 3 months ago

was this ever listed in the release notes?

It wasn't - I think it is time for Debbie to record another video on trace viewer.

in all honesty I feel like it's a much less user friendly experience.

We are trying to strike a balance here - there was an opposite push back suggesting that they wanted to see all the network requests regardless of the selected action. But more importantly, the network requests belonging to the action is a heuristic. The world is async, so these requests are picked based on timing rather than formal initiator / causality. So we think that making it controlled by the time window above + allowing to quickly select window for action is a more fair game.

ben-redd commented 3 months ago

Well, while I can't say I like the change for my use case, with time I'll probably get used to it and adapt. Thanks for taking the time to explain the reasoning behind the change, I really appreciate it 🙏

ryanrosello-og commented 3 months ago

I have triaged countless failed tests against a complex codebase with this feature. With it gone in the latest versions of playwright, I fear that debugging will become even more difficult.

@PsiKai I'm in the same boat :(

The trace viewer UX has dropped off since this change was removed.

debs-obrien commented 3 months ago

we did cover it in the release video showing UI mode which uses the trace viewer. The video goes into a lot of detail on all the new features:

https://youtu.be/YGJTeXaZDTM?feature=shared&t=575

but we can of course create a feature video that only focuses on these features and is more focused on trace viewer for those not using UI mode.

PsiKai commented 3 months ago

I'm going to close this issue. Apologies for the unproductive feedback. I will just have to adjust to the new viewer and stay up on ancillary updates such as the youtube video.