grafana / mimir

Grafana Mimir provides horizontally scalable, highly available, multi-tenant, long-term storage for Prometheus.
https://grafana.com/oss/mimir/
GNU Affero General Public License v3.0
4.17k stars 535 forks source link

Use resident set memory for the etcd memory alert. #9997

Closed seizethedave closed 13 hours ago

seizethedave commented 3 days ago

What this PR does

This changes the EtcdAllocatingTooMuchMemory alerts to use RSS memory rather than working_set. The reason for this is etcd can use a lot of file cache memory for its file-backed database that will be dropped if it approaches the container's memory limit. This causes this alert to basically be scare-noise, at times. We had a situation where the Go process only had 50MiB of memory resident, but there was 700MiB of file cache memory.

RSS fits better with "AllocatingTooMuch" anyway.

Which issue(s) this PR fixes or relates to

Fixes #

Checklist