Make a cached metric available at all times even when not reevaluated.
Current situation
With this configuration:
Metric is configured to be cached for 30mins
Prometheus scrape_interval is 5mins
Aggregation returns no document
overrideEmpty and emptyValue are configured accordingly to set value to 0
I only have 5mins of data in a 30min time window. It seems in this case that the metric is only exposed when cache expires and so when the aggregation is run again. I'm pretty sure these 5mins correspond to Prometheus query.loopback-delta configuration which means I only have one data point every 30mins and that Prometheus can only find it for 5mins.
Worth mentioning that if the aggregation do returns document I don't encounter the issue, it only happens when the override mechanism is triggered.
Should
Something is not working correctly in the scenario detailed previously and I believe it comes from the aggregate function where the metric creation differs when we don't have any document in the aggregation result. Metric should be available at all times even in the "overrideEmpty" case.
By the way it seems the DefaultCache is not acting as expected:
In this code I would expect that if you explicitly specify a different cache value from the default one on an aggregation it should not be overridden by the default one. Then, it should be == and not != so that an aggregation with no cache configured gets the cache value from the default one if it is configured.
Describe the change
Make a cached metric available at all times even when not reevaluated.
Current situation
With this configuration:
I only have 5mins of data in a 30min time window. It seems in this case that the metric is only exposed when cache expires and so when the aggregation is run again. I'm pretty sure these 5mins correspond to Prometheus query.loopback-delta configuration which means I only have one data point every 30mins and that Prometheus can only find it for 5mins.
Worth mentioning that if the aggregation do returns document I don't encounter the issue, it only happens when the override mechanism is triggered.
Should
Something is not working correctly in the scenario detailed previously and I believe it comes from the aggregate function where the metric creation differs when we don't have any document in the aggregation result. Metric should be available at all times even in the "overrideEmpty" case.
By the way it seems the DefaultCache is not acting as expected:
In this code I would expect that if you explicitly specify a different cache value from the default one on an aggregation it should not be overridden by the default one. Then, it should be
==and not!=so that an aggregation with no cache configured gets the cache value from the default one if it is configured.