Limit concurrency according to Linux container CPU quota by default

orestisfl commented 1 week ago

Welcome

[X] Yes, I've searched similar issues on GitHub and didn't find any.
[X] I agree to follow this project's Code of Conduct

Your feature request related to a problem? Please describe

golangci-lint supports automatically limiting the CPU quota if you use the --concurrency=0 flag / run.concurrency: 0 option.

However, the concurrency is set to runtime.NumCPU() and not 0 by default: https://github.com/golangci/golangci-lint/blob/225f7a06f26901f66539bf46fb04206cf53fc192/pkg/commands/run.go#L597.

As a result, golangci-lint will run by default with the number of detected logical cores instead of the limit. We observed that running golangci-lint on k8s resulted in it using 32 CPUs even though only 4 were available according to the limit set to the pod. That resulted in increased memory usage probably because the different threads were starved for CPU and were allocating memory in parallel. Eventually, the pod was OOM-killed after reaching the 8GB limit. By manually setting -j 4 or -j 0 we confirmed that the memory usage dropped by 50% and execution time was faster as well.

Describe the solution you'd like

Switch the default behavior to be equivalent to today's -j 0 behavior.

Describe alternatives you've considered

Manually setting the desired concurrency.

Additional context

No response

Supporter

[ ] I am a sponsor through GitHub or OpenCollective

ldez commented 1 week ago

Hello,

for now, we don't plan to change the default as it's a breaking change.

orestisfl commented 1 week ago

Thanks @ldez. FMI, what's the usecase that would break? From my perspective, respecting CFS limits is a desired behavior.

golangci / golangci-lint