microsoft / playwright

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
https://playwright.dev
Apache License 2.0
66.77k stars 3.66k forks source link

[Feature]: Allow for links in the attachment section of the playwright html reports #31284

Closed williamlabrum closed 3 months ago

williamlabrum commented 4 months ago

🚀 Feature Request

Ability to supply link instead of body or path. This would allow for folks to attach links on a per test run basis. This is an alternative to the existing Annotation.push() method that contextualizes these links at the test level instead of the needed test run level.

Example

Option 1: new contentType

this.testInfo.attach('Playwright Homepage', {
    contentType: 'text/html',
    body: `<a href="https://www.playwright.dev/" target="_blank">Playwright Homepage</a>`,
});

Option 2: link type

this.testInfo.attach('Playwright Homepage', {
    link: 'https://www.playwright.dev/',
    target?: '_blank', (_blank for default but could supply self if same window)
});

Option 3: extend path type (this seems less desirable as there is a lot of sanitization going on in this route)

this.testInfo.attach('Playwright Homepage', {
    path: 'https://www.playwright.dev/',
    target: '_blank',
});

Motivation

My company is able to provide a unique id that can be used to trace requests in DataDog APM tracing. The ability to have this link just makes it easier for the end user to click on a link and get to a particular trace for a particular test run. Annotations are an existing way to create links, but those are at the test case level, not the test run level so the contextualization is a bit harder to tie to a test run.

In summary: Having external links may be valuable at the run level (much like attachments) to facilitate additional observability.

yury-s commented 4 months ago

Folding into https://github.com/microsoft/playwright/issues/30095

yury-s commented 4 months ago

Sorry, I jumped the gun and folded it. Would annotations work for your use case if we showed the annotations per retry (currently it's always from the last run)?

test.info().annotations.push({ type: 'link - ' + it.info().retry, description: 'https://<data log url>' });
yury-s commented 4 months ago

Capturing discussion notes. Here is how we want to consider annotations vs attachments:

(might be related to https://github.com/microsoft/playwright/pull/31521)

williamlabrum commented 4 months ago

So the use case that I'm working through is that we create a unique correlation id (trace id) in Datadog's APM. This gives us the ability to trace all requests for a given test run. Since a retry is technically a new session, this Trace ID is renewed at the beginning of a new session. To me, it makes sense to put it in the attachments section. So we do that right now, however the link is un-clickable. So I've been looking for ways to make that link there clickable, instead of a copy/paste. Its small UX improvement to provide clickable links but thus far the annotations is the only place that I've been able to create that. Unfortunatly, the context on a given runtime link is hard to correlate to a given test run/retry. Agree with your definition above, just looking for a way to keep that definition yet get a better UX in attachments besides just plain text, or local file attachments (ie. zipfile or screenshot).

Would annotations work for your use case if we showed the annotations per retry (currently it's always from the last run)?

Given that your definition is static, part of the test definition, don't depend on the retry i wouldn't want to go against that. Contextualization i think is important, hence the suggestion of extending attachments. Just my 2c.