grafana / loki

Like Prometheus, but for logs.
https://grafana.com/loki
GNU Affero General Public License v3.0
22.75k stars 3.31k forks source link

Split multi-tenant query into tenant in query-frontend #13265

Open KimMJ opened 2 weeks ago

KimMJ commented 2 weeks ago

Is your feature request related to a problem? Please describe. Current behavior is iterate tenants in querier level. So total execution time will be SUM(duration of each tenant). This makes multi-tenant query slower if there are about 10 tenants.

Describe the solution you'd like A clear and concise description of what you want to happen.

Rather than iterate tenants in the querier level, split query into matched tenant in frontend level. queriers get task from the queue (query scheduler), and run for only one tenant. query-frontend will get results from queriers, and aggregate them as today it does.

Now total execution time will be MAX(duration of each tenant)

Describe alternatives you've considered Use single tenant rather than multiple tenants (ex. 10K). However, we cannot configure limits per tenant anymore.