search

grafana / k6-operator

An operator for running distributed k6 tests.
Apache License 2.0
541 stars 147 forks source link

Aggregate metrics results of TestRun #413

Closed or3330 closed 2 weeks ago

or3330 commented 4 weeks ago

Feature Description

As part of my usage in k6-operator, I wanted to get as a result of a TestRun my custom metrics and use it as part of the validations in my integration test flow.

I noticed that aggregated metrics from all of the TestRun pods on the operator manager level is not available, and I think it's a pretty important feature.

I will be happy to take this on!

Suggested Solution (optional)

I thought about using some sort of shared volume to send the metrics results of each pod as files, and then aggregate them on the manager level when the test finishes, and create a file with the name of the TestRun under some predefined folder on the manager pod volume.

Already existing or connected issues / PRs (optional)

No response

yorugac commented 4 weeks ago

Hi @or3330, it is an important feature but it's outside of scope for k6-operator itself. We currently provide aggregation as part of our Grafana Cloud k6 (GCk6). More on that can be read here: https://grafana.com/docs/grafana-cloud/testing/k6/

Specifically with k6-operator, there are two modes supported by GCk6:

Would those work for you?

or3330 commented 4 weeks ago

Hi @yorugac, thanks for the quick response!

I actually wanted to self host the solution, and not use an external provider.

yorugac commented 2 weeks ago

@or3330, sorry for the delay. I understand the problem but it is outside of k6-operator scope. The only way I can see it kind of partially supported by us is via distributed mode in k6 (issue) but it is still quite far away and most of that work will be outside of k6-operator anyway.

As mentioned, our solution for aggregation is cloud-based and not self-hosted. Of course, it is possible to set something self-hosted as k6 allows to stream its metrics to external DBs, like Prometheus. But it will not be a trivial task, and k6-operator project is simply not meant for that.

I'm closing the issue. Feel free to open another if there's a k6-operator specific question and / or comment in k6 repo about aggregation.