thanos-io / promql-engine

Multi-threaded PromQL engine implementation based on the Volcano paper.
Apache License 2.0
142 stars 54 forks source link

Support query stats #148

Open yeya24 opened 1 year ago

yeya24 commented 1 year ago

The original promql engine has builtin support for query stats https://github.com/prometheus/prometheus/blob/main/util/stats/query_stats.go#L112.

It has mainly 2 parts: timings and samples.

// queryTimings with all query timers mapped to durations.
type queryTimings struct {
    EvalTotalTime        float64 `json:"evalTotalTime"`
    ResultSortTime       float64 `json:"resultSortTime"`
    QueryPreparationTime float64 `json:"queryPreparationTime"`
    InnerEvalTime        float64 `json:"innerEvalTime"`
    ExecQueueTime        float64 `json:"execQueueTime"`
    ExecTotalTime        float64 `json:"execTotalTime"`
}

type querySamples struct {
    TotalQueryableSamplesPerStep []stepStat `json:"totalQueryableSamplesPerStep,omitempty"`
    TotalQueryableSamples        int64      `json:"totalQueryableSamples"`
    PeakSamples                  int        `json:"peakSamples"`
}

// BuiltinStats holds the statistics that Prometheus's core gathers.
type BuiltinStats struct {
    Timings queryTimings  `json:"timings,omitempty"`
    Samples *querySamples `json:"samples,omitempty"`
}

It would be nice to support capturing those stats in this query engine.

yeya24 commented 1 year ago

Since this query engine is different from the old engine in terms of the query execution stages, I feel we don't have to support query timings for now. But it would be nice to support query samples first.

fpetkovski commented 1 year ago

Can we solve this through https://github.com/thanos-community/promql-engine/issues/106 ?

yeya24 commented 1 year ago

I think #106 has a larger scope than this one. This issue is mainly to support Prometheus query stats in the engine because currently if I enable query stats the engine panics.

fpetkovski commented 1 year ago

Got it, makes sense.

sahnib commented 1 year ago

I would like to contribute to adding support for query stats in the engine.