grafana / alloy

OpenTelemetry Collector distribution with programmable pipelines
https://grafana.com/oss/alloy
Apache License 2.0
1.47k stars 221 forks source link

[pyroscope.java] Old JFR files accumulating on disk #1960

Open swar8080 opened 1 month ago

swar8080 commented 1 month ago

What's wrong?

When alloy is terminated, JFR files written to disk by the pyroscope.java file never seem to be cleaned-up. In an extreme scenario when alloy is frequently restarted, like if OOMKilled on kubernetes, then maybe the files gradually accumulate and start filling up the host's disk.

This is making us take a gradual approach to adopting continuous profiling since we're worried about disk pressure on our kubernetes nodes. Not sure how valid of a concern that is, but it would be nice if alloy cleaned-up any of old JFR files, like during start-up

Steps to reproduce

System information

Ubuntu Linux 22.04.3 x86_64

Software version

v1.4.3

Configuration

https://github.com/grafana/pyroscope/blob/main/examples/grafana-agent-auto-instrumentation/java/kubernetes/grafana-alloy.yaml

Logs

No response

github-actions[bot] commented 17 hours ago

This issue has not had any activity in the past 30 days, so the needs-attention label has been added to it. If the opened issue is a bug, check to see if a newer release fixed your issue. If it is no longer relevant, please feel free to close this issue. The needs-attention label signals to maintainers that something has fallen through the cracks. No action is needed by you; your issue will be kept open and you do not have to respond to this comment. The label will be removed the next time this job runs if there is new activity. Thank you for your contributions!