memory leak in 0.11.0 and 0.12.0 ?

[frankhetterich]

Hello, we updated last week our Eventstore Exporter Pods to version v0.11.0. After the upgraded we noticed that the Containers are using a lot more memory than before and also reaching the memory limit of 30MB and getting OOM killed. We updated this week then to v0.12.0 and had the same issues. Now we´re downgraded to v0.10.4 and everything is fine again.

Memory usage of v0.11.0

Memory usage of v0.10.4

regards, Frank

[marcinbudny]

Hi Frank,

Since version 0.11.0, the exporter uses the ESDB golang client for some of the operations. It is possible that a memory leak was introduced with this change.

Please provide me with additional details:

• what is the exact configuration of the exporter you're using (e.g. do you use stream and subscription statistics)?
• what version of ESDB you're using and do you run it in a cluster?
• other than memory leak, is the exporter functioning correctly and returning metrics as expected?
• are there any errors logged

[frankhetterich]

Please provide me with additional details:

• what is the exact configuration of the exporter you're using (e.g. do you use stream and subscription statistics)?

I think it´s the default config, no special modifications

clusterMode=single enableParkedMessagesStats=false eventStorePassword="**REDACTED**" eventStoreURL="http://**************" eventStoreUser=***** insecureSkipVerify=true port=9448 timeout=20s verbose=true

• what version of ESDB you're using and do you run it in a cluster?

We´re running different Versions of Eventstore 20.6 and 20.10. The instances are running as single but also as cluster (depends on the environment)

• other than memory leak, is the exporter functioning correctly and returning metrics as expected?

Everything is running fine, all metrics are available in the Grafana Dashboards

• are there any errors logged

I couldnt find any errors in the container logs

[marcinbudny]

Probable culprit: https://github.com/EventStore/EventStore-Client-Go/issues/113

[marcinbudny]

I released version v0.12.1 which should not leak memory in your configuration. With v0.12.1, ESDB go client is not created if both parked message stats and stream stats are disabled.

Waiting for a reply from ESDB go client team to apply a proper fix.

[frankhetterich]

Deployed V0.12.1 this morning. Memory consumption looks good so far.

Thanks for the quick fix

[marcinbudny]

The v0.12.2 release uses the latest ESDB client version that fixed the memory leak. It looks good in my tests. Let me know if you experience any further issues.