[Feature] A proper way to display playwright-html-reports after running a playwright test as a build-step

[harun911]

Playwright offers some fantastic features for debugging (html testresults, trace.zip), but we are having a hard time displaying these results inside Azure Devops. Currently, our workflow is: navigating to failed build, downloading artifacts (video, screenshots, trace.zip), opening screenshots and video 1 by 1 and then navigating to trace.playwright.dev, uploading the trace.zip and then start debugging. We would like to be able to navigate to the html-testresults and/or playwright trace viewer with a single click from the build summary. This makes it much easier for all of our team members and organization and keeps our workflow clean and neat, reducing unnecessary steps. Thanks so much for all the great work!

[kanchansharma79]

@harun What is the best approach to set trace-viewer Path so that we can get the file in our build pipeline.

I am getting below error: 2023-02-25T04:23:59.4906784Z System.ArgumentException: Path cannot be the empty string or all whitespace. (Parameter 'path') 2023-02-25T04:23:59.4907526Z at System.IO.Directory.CreateDirectory(String path) 2023-02-25T04:23:59.4908361Z at Microsoft.Playwright.Core.Artifact.SaveAsAsync(String path) in //src/Playwright/Core/Artifact.cs:line 69 2023-02-25T04:23:59.4909356Z at Microsoft.Playwright.Core.Tracing.DoStopChunkAsync(String filePath) in //src/Playwright/Core/Tracing.cs:line 103 2023-02-25T04:23:59.4910354Z at Microsoft.Playwright.Core.Tracing.<>cDisplayClass9_0.<b_0>d.MoveNext() in //src/Playwright/Core/Tracing.cs:line 66 2023-02-25T04:23:59.4912879Z --- End of stack trace from previous location --- 2023-02-25T04:23:59.4913956Z at Microsoft.Playwright.Transport.Connection.<>cDisplayClass45_0.<b_0>d.MoveNext() in //src/Playwright/Transport/Connection.cs:line 494 2023-02-25T04:23:59.4914893Z --- End of stack trace from previous location ---

[jarodsmk]

I'd also like to advocate for a feature like this ⭐

On enterprise-level projects that require showing feedback to non-technical staff & management, it would be ideal to have something that could display what's already prepared by the HTML reporter, without having to install dependencies and run the show-report command, or some means of displaying it on an agent.

For the case of Azure agents, running the show-report command after a test run keeps the agent running indefinitely, until the timeout of the pipeline reaches. So while there would be ways to get the HTML report displayed by mapping the static IP of an on-premise agent using the Playwright settings to do so, it's still not a feasible solution

[Az8th]

Hello, I have the same need, and have good new for you ;) I found a partial response by browsing other issues and discovered that you can provide the web trace viewer the path of your trace.zip like so : https://trace.playwright.dev/?trace=https://alisterscott.github.io/media/trace.zip

Then, by adding your report to pipeline artifacts and retrieve its download URL, you should be able to load it in the web viewer directly.

The only problem I have to get this working is that the viewer does not recognize my file as valid (?trace=https://[AzDO_URL]/[COLLECTION]/[PROJECT]/_apis/resources/Containers/[ID]/Test-logs?itemPath=Test-logs%2Ftrace&%24format=zip&saveAbsolutePath=false). But it may be due do the fact the trace is actually badly formed, or not compressed within the artefact, I will investigate further, but feel free to test it by yourself and tell me if it works ;)

[Az8th]

Update : I had an invalid trace.zip file, which is fixed, but I still I can't open it with the web trace viewer using git or artifacts download URLs from Azure DevOps. I suspect a missing CORS policy (see #104555, especially this comment)

[Az8th]

I found a usable workaround by using the dashboard view with the iframe widget to target the HTML report produced by our Playwright CI (it is hosted on another server until I can figure out how to access it directly from pipeline artifacts, or via API without having to give a PAT token).

You can have access to all produced runs, including embedded videos or traces, or even make the plugin target the traces directly. Still, it is limited as you can't scroll or go back, but I will open a feature request shortly to ask for improvement ;)