search

grafana / k6

A modern load testing tool, using Go and JavaScript - https://k6.io
GNU Affero General Public License v3.0
25.91k stars 1.27k forks source link

Automatically set GOMAXPROCS and GOMEMLIMIT based on available resources (container cgroups / system) #3941

Open frittentheke opened 1 month ago

frittentheke commented 1 month ago

Feature Description

While k6 itself does take the number of CPUs into account, e.g. at https://github.com/grafana/k6/blob/f49504674362639fa312d9b5bd8ecf8d8f93ddbd/execution/scheduler.go#L268 (planned at https://github.com/grafana/k6/blob/f49504674362639fa312d9b5bd8ecf8d8f93ddbd/output/cloud/expv2/output.go#L116) the value has to be configured manually and explicitly.

So manual adjustments to GOMAXPROCS and GOMEMLIMIT are required to tune K6 to work best with the available resources of the system (hardware, VM, container).

I use k6s on Kubernetes and via the k6-operator and the provided TestRun CustomResource does allow for requests and limits to be configured. While these settings are then applied to the PodSpec, they are not converted or made available to the corresponding golang settings GOMAXPROCS and GOMEMLIMIT.

While this capability could be added to the k6-operator (shout out to @yorugac), to me it makes more sense to enable k6 itself to auto-tune itself to the available resources as it can be run without the k6 operator or non-containerized and will still benefit from these auto-tuning capabilities. There is a similar suggestion for the Grafana Helm chart, also discussing the use of auto-tuning for the options: https://github.com/grafana/helm-charts/pull/3138 (@jnoordsij, @jkroepke)

Therefore I'd like to propose adding automatic config of the Golang runtime via well established golang libs, which are actually commonly used for other Grafana tools:

GOMAXPROCS

GOMEMLIMIT

There are many examples / MRs for the proposed mechanisms / libraries for other popular tools ....

Suggested Solution (optional)

No response

Already existing or connected issues / PRs (optional)

No response

olegbespalov commented 1 month ago

@grafana/k6-core, what do you think?

joanlopez commented 1 month ago

@grafana/k6-core, what do you think?

I don't have specific experience with these libraries, but it definitely sounds as something that may worth looking at and trying out. Although it isn't probably something with really high priority, tbh.