search

neondatabase / neon

Neon: Serverless Postgres. We separated storage and compute to offer autoscaling, code-like database branching, and scale to zero.
https://neon.tech
Apache License 2.0
14.36k stars 414 forks source link

export jemalloc metrics, add (de)allocation counts #8656

Open koivunej opened 1 month ago

koivunej commented 1 month ago

With pageservers switch to jemalloc following proxy's lead, in a recent memory leak it came up that we should have metrics from jemalloc.

Proxy already has implemented reading metrics of jemalloc in https://github.com/neondatabase/neon/blob/main/proxy/src/jemalloc.rs.

We could first port the jemalloc metrics over to the legacy metrics utility (without disturbing proxy) or make a bridge between the implementations.

Later we could add the actual (de)allocation counts for example by sharded thread-local counters, as jemalloc does not seem to make this available by default. This part needs more investigation, is it error prone to provide such counting?

koivunej commented 1 month ago

Adding the good_first_issue for the first part; a bridging between the two might even already exist.