Disable per-index metrics in config.

[fulmicoton]

On some of our users with a lot of indexes, we ended exposing thousands of metrics. The endpoint is fast enough, ... but metric backend can suffer from the high cardinality.

Solution 1

Just add a disable_index_level_metrics to quickwit config, that would use the label "_any" for all index label in metrics.

Solution 2

One neat way we could do that for instance, could be by adding a index_metric_group parameter in the index config that would be used instead of the index in the metric label if supplied. (if not fallback to the index id)

It might not be a proper solution for the control plane metrics however. We could entirely remove the index labels for that one.

[esatterwhite]

The index level information is valuable / useful. If it could be exposed from an API endpoint, people could build their own tooling around it.

[guilload]

Closed via #5125.

[trinity-1686a]

the solution 1 has been implemented, do we want to eventually support solution2?