Enriched select hints

[fpetkovski]

Is your proposal related to a problem?

The hints which are extracted by the PromQL engine are hard to extend at the moment. Their use is also fairly limited since they treat windowing functions the same way as aggregations. Furthermore, because hints are a generic API for many different remote-read storage systems, they are fairly rigid and hard to change and evolve.

Examples where we are limited by query hints at the moment:

• Capturing both windowing and aggregation functions for query pushdown: https://github.com/prometheus/prometheus/issues/11703
• Inability to propagate parameters in aggregations like topk and bottomk : https://github.com/prometheus/prometheus/issues/9979
• Inability to implement projection pushdown based on existing hints: https://github.com/thanos-io/thanos/pull/5892
• I am currently looking into implementing a Thanos Store for the Stackdriver API. Stackdriver can do aggregations but requires a so-called alignment function to align timestamps from individual series. Most windowing functions from PromQL (rate, delta, *_over_time) can be translated to aligners in Stackdriver.

Describe the solution you'd like

I would like to propose extracting better hints in Thanos and propagating them to underlying stores through the Thanos Store API. One way to do it is to traverse the PromQL AST before execution and extract hints for the selectors. Hints are linked to selectors by injecting a special label which is stripped when doing the actual select call: https://github.com/thanos-io/thanos/pull/5983.

Describe alternatives you've considered

One alternative is to extend the select hints in Prometheus itself, but this seems to be quite challenging since reviewers lack context on how the additional fields will be used. Even something like https://github.com/prometheus/prometheus/pull/11163 took a very long time to get merged.

[fpetkovski]

I've raised https://github.com/thanos-io/thanos/pull/6012 which can solve some of the problems described in this issue.