cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.11k stars 3.81k forks source link

cgroups.go picking up memory.pressure mountpoint #112340

Open robdinh opened 1 year ago

robdinh commented 1 year ago

Describe the problem Since this commit was applied, systemd creates a cgroup2 mount for memory.pressure.

$ pid=$(sudo systemctl show --property MainPID cockroachdb | cut -d'=' -f2)
$ sudo cat /proc/$pid/mountinfo | grep cgroup2 | cut -d' ' -f5
/sys/fs/cgroup
/sys/fs/cgroup/production.slice/production-storage.slice/cockroachdb.service/memory.pressure

With ProtectControlGroups=yes option set, /pkg/util/cgroups/cgroups.go/ ends up detecting the memory.pressure path. This is causes a noisy log showing this message:

"message": "unable to get CPU capacity: error when read cpu quota from cgroup v2 at ‹/sys/fs/cgroup/production.slice/production-storage.slice/cockroachdb.service/memory.pressure/production.slice/production-storage.slice/cockroachdb.service/cpu.max›: open ‹/sys/fs/cgroup/production.slice/production-storage.slice/cockroachdb.service/memory.pressure/production.slice/production-storage.slice/cockroachdb.service/cpu.max›: not a directory"

where maxFilePath in detectCPUQuotaInV2() is a join of strings of:

To Reproduce This continue might be how /pkg/util/cgroups/cgroups.go/ picks other cgroup2 mounts, where it used to be a return.

Expected behavior maxFilePath should be /sys/fs/cgroup/production.slice/production-storage.slice/cockroachdb.service/cpu.max

Environment:

Jira issue: CRDB-32361

blathers-crl[bot] commented 1 year ago

Hello, I am Blathers. I am here to help you get the issue triaged.

Hoot - a bug! Though bugs are the bane of my existence, rest assured the wretched thing will get the best of care here.

I was unable to automatically find someone to ping.

If we have not gotten back to your issue within a few business days, you can try the following:

:owl: Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.