search

checkly / public-roadmap

Checkly public roadmap. All planned features, updates and tweaks.
https://checklyhq.com
37 stars 7 forks source link

Visual support for performance traces #221

Open pondorasti opened 2 years ago

pondorasti commented 2 years ago

Is your feature request related to a problem? Please describe. One of my use cases for ChecklyHQ is tracking custom metrics through the Performance API. My app leverages performance.mark() to create it's own performance KPI and then check for performance regressions. Inside my checks I'm creating performance traces and then programmatically reading trace.json to compute the KPIs. In addition to this, I would like to have access to my traces after a check finishes running.

await browser.startTracing(page, { path: “trace.json”, screenshots: true })
// ....
await browser.stopTracing()

Describe the solution you'd like Similar to how you can take a screenshot and see it as part of the test, I would love to have access inside a check's dashboard to the generated trace.json. In some ways, similar to how a screenshot is auto taken when a test fails, having access to the trace will help debug and understand what went wrong.

Describe alternatives you've considered Only viable alternative I see is to have a snippet run at the end which uploads the trace to a storage bucket and posts the url in the console.

Additional context N/A

tnolet commented 2 years ago

@Pondorasti thanks for bringing this. We are diving into full playwright/test support soon and will look at supporting more of the assets generated by PW. Tagging @darkostanimirovic here too

darkostanimirovic commented 1 year ago

Hi @Pondorasti. We have released support for Playwright Test traces recently. Any time a check fails, you'll have the trace and video on the check run results page. On the browser check edit screen, you have the same but any time you run/preview your check.

While this won't give you direct access to the recorded trace.json as you described, I thought you might be interested.

darkostanimirovic commented 1 year ago

Maybe more importantly, in 2023 we will be working on support for custom metrics. Theperformance API is one possible route we might take, but there's no clear spec or ETA as of now. Can you describe what a "perfect scenario" would look like for you when using the performance API?