grafana / loki

Like Prometheus, but for logs.
https://grafana.com/loki
GNU Affero General Public License v3.0
23.58k stars 3.41k forks source link

Implement filters for the Ruler `api/v1/rules` prometheus compatible API #9295

Closed gotjosh closed 8 months ago

gotjosh commented 1 year ago

Is your feature request related to a problem? Please describe.

https://github.com/grafana/grafana/issues/66701

In Grafana, a user has the ability to view the information of a single rule. When this page is loaded Grafana calls the api/v1/rule which pulls all of the state of rules of that the tenant has.

I've recently implemented filtering by rule name, rule group and file (namespace in the case of mimir) in Prometheus. We should do the same for Loki.

Describe the solution you'd like

The rules API does a fanout request to all the other rulers of which the tenant has rules and then aggregates the results before creating the response.

We have two options:

  1. We can do the filtering after we're aggregated the results - it's a shorter implementation path but it means we'll still transmit all of the data across the network to then throw it away.
  2. We pass the filters down to the GRPC requests to limit the data returned - it's a bit of a larger effort but gets us a significant reduction of network bandwidth given that this query is called quite often (every 20s) per client using the alerting view in Grafana.

I'd argue we should go for option 2, given that there's a plan to implement pagination and setting a foundation for filtering the data transmitted between rulers will become even more useful.

The Prometheus equivalent changes are https://github.com/prometheus/prometheus/pull/12270

Describe alternatives you've considered

N/A

Additional context

N/A

### Tasks
- [ ] Change the proto definition of `GroupStateDesc` to allow filtering options by `rule_name`, `rule_group` and `file[]` (namespace in the case of mimir)
- [ ] Allow support for these query filters in the Prometheus compatible endpoint API
- [ ] Document the supported filters

The mimir equivalent of this is in: https://github.com/grafana/mimir/issues/4792

aminesnow commented 1 year ago

Hey @gotjosh :wave: I've opened a PR ( #9128 ) about adding a filter on rules labels, and it seems to be in the same spirit of what you're proposing. Please let me know what you think about it and if it's inline with what you had in mind!

gotjosh commented 1 year ago

Hi @aminesnow, sadly it isn't what you have implemented is what I described as option 1 and I think the implementation should follow a bit more of what I think it's option 2.

aminesnow commented 1 year ago

So if I understand correctly, you're suggesting we pass the filters down to the store and do the filtering there, right?