Tracking peak samples in engine

thanos-io / promql-engine

Multi-threaded PromQL engine implementation based on the Volcano paper.

Apache License 2.0

131 stars 52 forks source link

Tracking peak samples in engine #452

Closed pedro-stanaka closed 1 month ago

pedro-stanaka commented 1 month ago

Summary

Right now the engine will report peak samples (in a single step) and total samples always as the same, because we are using the method IncrementSamplesAtStep which will update peak and total with the same value. In this PR I am using both values and calling methods on the stats.QuerySamples struct directly.

fpetkovski commented 1 month ago

The remote execution operator uses a vector selector underneath. Could we extract stats from there?

fpetkovski commented 1 month ago

I guess the question is whether we want to eventually bubble up processed samples from remote engines, but this should fix the bug for now.

pedro-stanaka commented 1 month ago

Pushed one more commit that will properly consume sample statistics from downstream query, it makes bubbling up sample information more precise. Tested with changes from https://github.com/thanos-io/thanos/pull/7361