An issue was reported in #228 where histograms did not appear to be removed after they exceeded the configured idle timeout.
There were two bugs, one in the version the user reproduced the issue with and one in the current version of the code:
used wrong method to check for/delete idle histograms (current version)
did not remove aggregated distributions for histograms that were reported as inactive (both versions)
Primarily, the issue centered around the fact that the exporter aggregated histograms (distributions) internally every time something wants to render the metrics. This provided bounded memory usage over time for a given metric, as well as tracking aggregated histograms/summaries in the way that Prometheus expects.
However, even though we would correctly delete a histogram from the registry when we detected that it had been idle too long, we did not delete the internally aggregated version that the exporter also held... and since rendering works off of the internally aggregated data, it essentially ended up as the histogram continuing to be rendered/reported.
An issue was reported in #228 where histograms did not appear to be removed after they exceeded the configured idle timeout.
There were two bugs, one in the version the user reproduced the issue with and one in the current version of the code:
Primarily, the issue centered around the fact that the exporter aggregated histograms (distributions) internally every time something wants to render the metrics. This provided bounded memory usage over time for a given metric, as well as tracking aggregated histograms/summaries in the way that Prometheus expects.
However, even though we would correctly delete a histogram from the registry when we detected that it had been idle too long, we did not delete the internally aggregated version that the exporter also held... and since rendering works off of the internally aggregated data, it essentially ended up as the histogram continuing to be rendered/reported.
Fixes #228.
Signed-off-by: Toby Lawrence toby@nuclearfurnace.com