search

PromPHP / prometheus_client_php

Prometheus instrumentation library for PHP applications
https://prometheus.io/docs/concepts/metric_types/
Apache License 2.0
415 stars 91 forks source link

APCng storage - avoid worst-case memory usage in buildPermutationTree #123

Open Princemachiavelli opened 1 year ago

Princemachiavelli commented 1 year ago

The peak memory usage of the APCng storage adapter while collecting the metrics for display is currently always the worst-case for the cardinality of the metric label values.

This change replaces the buildPermutationTree implementation with a (cartesian product) generator which avoids keeping every metric label value permutation in memory at once. This can reduce peak memory usage by orders of magnitude for metrics with high cardinality but sparse actual usage. This helps avoid exceeding the PHP memory_limit.

Besides the reduced memory usage, I haven't seen much of a difference in how long buildPermutationTree takes to run. (To improve that, only the actually used metric label pairs could be to tracked separately to avoid building the permutations completely.)

mlebrun commented 1 year ago

This would be great to land, we are hitting the PHP limit as well.