pganalyze / collector

pganalyze statistics collector for gathering PostgreSQL metrics and log data
https://pganalyze.com
Other
324 stars 54 forks source link

Host resources vs Pod resources #195

Open GeertJohan opened 2 years ago

GeertJohan commented 2 years ago

I'm running the collector as a sidecar container in a zalando/postgres-operator managed postgres cluster. It seems that the reported machine specs (cpu, memory) are those of the host. I think it makes more sense if the collector would report the cpu and memory for the container it's managing. Or maybe the collector could have override flags so that I can tell it what the correct values are.

Right now I get a warning about shared_buffers because the host machine has a lot more memory than the pod. So the shared_buffers ratio when comparing with the host is wrong. But it's actually set to 25% when compared with the containers's max memory.

lfittl commented 2 years ago

@GeertJohan Yes - thats a good point, and a known issue in a sense.

We'll have to adjust the logic we use for retrieving OS statistics to be cgroup-aware, i.e. reading from the /sys/fs/cgroup folder. The library we currently use for reading system statistics (gopsutil) has some support for this, but it seems the author would like to remove this at some point: https://github.com/shirou/gopsutil/pull/996#issuecomment-727768410 -- the library also does not seem to correctly support ECS containers: https://github.com/shirou/gopsutil/issues/1061

Ideally we would utilize a different library for getting the cgroup statistics, but a quick search did not turn up something usable, so we'll have to investigate a bit more, or write our own. Thoughts welcome, in case you are familiar with the Go libraries around this.

Overall better Kubernetes support is on our near-future roadmap, and I'll include a reference to this issue in our planning document. We'll update this issue once we make progress / implement a solution.