grafana / xk6-browser

k6 extension that adds support for browser automation and end-to-end web testing via the Chrome Devtools Protocol
https://grafana.com/docs/k6/latest/javascript-api/k6-experimental/browser/
GNU Affero General Public License v3.0
337 stars 42 forks source link

Group Response Time on "FAIL or ERROR" #544

Closed LBouet closed 1 year ago

LBouet commented 1 year ago

Hi All,

When an error occurs in the group and the exception is thrown with a Check = FAIL. The "group_duration" returns a value. No status is assigned to the group_duration to filter out the "fails".

By counting the "fail" response times, the overall response times are wrong.

During a long duration load test, it is allowed to have an acceptable error rate. Indeed, the overall response times can be improved or totally degraded.

It is therefore impossible for us to graph the results of the response times of the groups.

We need to set up a "Timers" (Trend) to record these response times.

The solution to lighten the code, would be to add a status field to the group_duration or no value.

Thanks

imiric commented 1 year ago

Hi there,

this topic is better suited for our community forum, where we can discuss your issue and decide whether it's something worth tracking in our GitHub issues. See our SUPPORT document. I'll answer you partly here, but you should create a forum thread instead.

group_duration is a metric emitted by k6, not xk6-browser. This extension has no control over the group() functionality, so we should involve k6 core developers in this discussion, which is another reason why the forum is a better place for this.

In general, though, there is no concept of "group response time". A k6 group is a function that can execute anything, not just HTTP requests or checks, so it doesn't make sense for it to have a "status" field. It would probably be convoluted to add such an integration in the k6 codebase anyway. This is why the only thing it tracks is the group execution time.

We need to set up a "Timers" (Trend) to record these response times.

Right, that would be a better way of doing this. You could also track the request response time of "failed" requests using the http_req_failed metric, and correlating the URL with http_req_duration, but not sure if this would work for your use case.

So since there's a valid workaround for this, I doubt something like what you suggest would get implemented in k6, but feel free to create a forum thread and we can discuss it with the rest of the team. Use the "OSS Support" sub-forum if you do.