search

VictoriaMetrics / operator

Kubernetes operator for Victoria Metrics
Apache License 2.0
422 stars 140 forks source link

VictoriaMetrics operator should be cgroup aware #875

Closed uhthomas closed 1 month ago

uhthomas commented 6 months ago

Go is not cgroup aware. This means the VictoriaMetrics operator will be throttled hard in containerised environments with CPU limits.

The below screenshot shows the container with a CPU limit in Kubernetes. The first half of the time series is on a 13600k, with 20 threads. The second half is when the machine changed to use an EPYC 7763 with 128 threads.

image

automaxprocs can use cgroups automatically.

Haleygo commented 6 months ago

Hello @uhthomas , thanks for the issue! AFAIK the operator code itself doesn't use muti-thread to process resource objects, so I'm not sure the performance benefit here. Did you see some valid results with new GOMAXPROCS?

f41gh7 commented 1 month ago

Hello, thanks for reporting the issue.

Currently, operator doesn't use GOMAXPROCS for paralleling CPU bound tasks. Actually, it doesn't perform any heavy CPU bound operations. And unlikely it will change in the future.

Meanwhile the issue itself looks to me more like golang runtime problem, neither something that each application should fix on it's own.