vitalets / playwright-bdd

BDD testing with Playwright runner
https://vitalets.github.io/playwright-bdd/
MIT License
290 stars 34 forks source link

Open Playwright trace from HTML report without downloading #103

Closed vitalets closed 2 days ago

vitalets commented 6 months ago

Hi @vitalets, the html report looks great. One thing I wanted to ask if you can make it possible the traces of a failed test to be opened directly within the browser with a click. Currently I need to download the trace file and open it with command 'npx playwright show-trace ....'.

Within TeamCity when I am using the playwright html report I also have the option to just click it and it will open in a frame, so no need to download it.

image

image

this is sample from the html how it is opened in TeamCity iframe

image

Maybe passing the URL of the traces could help

image

Originally posted by @NikkTod in https://github.com/vitalets/playwright-bdd/issues/9#issuecomment-1983940651

vitalets commented 6 months ago

Interesting idea, should think about it. The main point here is that trace.zip should be uploaded to some public storage to be opened with trace.playwright.dev.

There are two questions:

  1. Could you check, that artefacts from your teamcity server are available to trace.playwright.dev? I mean checking that url like this https://trace.playwright.dev/?trace=https://you-teamcity-server/../trace.zip shows trace?
  2. if you use Playwright's original HTML reporter, do you still need to download trace.zip to open it? If yes, do you use attachmentsBaseURL option?
NikkTod commented 6 months ago

Hi @vitalets, we should have a look at how playwright team are making the remote view traces available directly within their html report.

image

I had a look in TeamCity on our side and this is what I see:

The view option in the playwright's original HTML reporter generates the following link:

https://our-teamcity-server:id/reports/trace/index.html?trace=https://our-teamcity-server:id/reports/data/traceID.zip

If I copy and paste this link in a new browser tab, the traces are opened directly within that tab.

Just for your information the only thing that I am configuring in TeamCity is to point to the path where the html playwright report is being generated.

If I open in a browser tab only the second part of the above link it will just download the zip file -> https://our-teamcity-server:id/reports/data/traceID.zip

So it seems that they do not use in their html report view the https://trace.playwright.dev/ url, but the following structure https://remote-server:id/reports/trace/index.html?trace=.....

So from all above observations let me try to answer the questions:

  1. I am not able to open the traces if I use the trace.playwright.dev url. It gives me an error "Could not load trace from https://our-teamcity-server:id/reports/data/traceID.zip" Make sure a valid Playwright Trace is accessible over this URL." But if I paste that https://our-teamcity-server:id/reports/data/traceID.zip in a browser tab the zip is being downloaded successfully.

  2. When using the Playwright's original HTML reporter I do not download the traces zip, I directly open the traces from the html report. I am not using attachmentsBaseURL

NikkTod commented 6 months ago

@vitalets This is how the html traces link looks for localhost

localhost:port/trace/index.html?trace=https://localhost:port/data/traceID.zip

vitalets commented 6 months ago

Nice investigation. I didn't realize that Playwright HTML report also acts as trace viewer. I've check Playwright HTML reporter source, they generate trace url this way:

export function generateTraceUrl(traces: TestAttachment[]) {
  return `trace/index.html?${traces.map((a, i) => `trace=${new URL(a.path!, window.location.href)}`).join('&')}`;
}

What we can do for Cucumber reports:

  1. output Playwright HTML report along with Cucumber HTML report
  2. generate trace links in Cucumber report redirecting to Playwright HTML report

Another option is to embed Playwright trace viewer into Cucumber report somehow.

One more thing I've noticed - traceviewer works only on http / https urls, on file:// scheme it shows an error: image For Cucumber reports on Teamcity it will be ok (as they are on https://, for opening report files locally it will not work.

Lets think more about it.

NikkTod commented 6 months ago

Embeding the Playwright trace-viewer into Cucumber report sounds great.

As you said, currently you can not open the traces from the playwright html report locally, it gives the above error, but I don't think this is a stopper, because locally most of the users are using directly trace.playwright.dev.

In my case, because of company restrictions, the trace.playwright.dev url is blocked, so if needed locally I am using the 'npx playwright show-trace path/to/trace.zip'

Maybe we can create a feature ticket on playwright side to allow also file://

arieluchka-bse commented 3 months ago

adding a flag to create an "All-in-one" tracer zip would also be amazing! :) being able to send the tracer zip and knowing that the receiving end can watch it without installing playwright/going to trace.playwright.dev

vitalets commented 3 months ago

@arieluchka-bse Just to clarify, under 'all-in-one' tracer you mean Cucumber report bound with tracer, right? So you can generate cucumber report in some directory, zip this directory and send to someone?

arieluchka-bse commented 3 months ago

hi, apologies if it's the wrong repo to comment on. im fairly new to playwright and testing in general, and i've never heard about cucumber report. im using playwright with python, so for test reporting im using pytest. what i meant by "all-in-one" is just for the tracer. generating a tracer.zip with all html files that are needed to see the trace, without depending on the trace.playwright.dev.

from what i saw, in NodeJS there is an option to set the "reporter" to html, and all the test results + traces, can be opened locally with the generated reporter (so no need to navigate to the site.

i wanted to implement something similar to what's showing here, with providing links to trace.zips, but the author of this achived it with setting the reporter to html (tutorial).

just wish there was something like this for python, that will allow viewing traces, just by opening the zip file (without downloading or navigating anywhere else)

vitalets commented 3 months ago

There is Playwright's CLI command show-trace, does it serve your needs?

npx playwright show-trace path/to/trace.zip
arieluchka-bse commented 3 months ago

hi, what i meant is being able to share traces with co-workers that do not have playwright installed.

in Node, if we set reporter=html, the test results are outputed into a zip that is accessed as an html file.

this makes it very easy to send the results to people who do not have playwright installed (they can just open the html file, and see all test results and traces, without needing https://trace.playwright.dev/ to view the trace, and without installing playwright locally).

This functionality exists in node, but not in python :(

vitalets commented 3 months ago

@arieluchka-bse I agree such case is useful. One note here: to see the trace people need to open html report via https, not just clicking to index.html and open on file:// protocol. Anyway, embedding trace-viewer into cucumber report is good feature and is planned in playwright-bdd.

vitalets commented 2 days ago

Trace view in Cucumber HTML report is released in Playwright-bdd v7.5.0. Feel free to open new issues in case of any errors.