thanos-io / promql-engine

Multi-threaded PromQL engine implementation based on the Volcano paper.
Apache License 2.0
141 stars 54 forks source link

Calculate instant query count() without fetching samples #143

Open GiedriusS opened 1 year ago

GiedriusS commented 1 year ago

See: https://github.com/prometheus/prometheus/issues/10073

If it's an instant query then we can calculate count() without fetching any samples with some caveats:

Now I think about it some more, I'm not sure we can get the same result without samples.
I think in the current code it will look back the stale time (5 minutes), and if there is no sample then the series will not appear.
Whereas the alternative code path will return every series in the head, typically anything that had a sample in the last 2-3 hours.

So while I still want the cheaper implementation, we would need to address backwards-compatibility.
GiedriusS commented 1 year ago

Hmm, I wonder how it would work if we called /api/v1/series with the respective matchers and start/end set to the same timestamp. Does it mean that the /api/v1/series is broken if it returns series even outside the requested timestamp i.e. series that is in the head but does not necessarily exist during the given time instant? :smile: need to double-check

MichaHoffmann commented 4 months ago

I think this is not well defined because of lookback maybe! You would still need to decode the chunks to check if there is a sample within lookback period