grafana / mimir

Grafana Mimir provides horizontally scalable, highly available, multi-tenant, long-term storage for Prometheus.
https://grafana.com/oss/mimir/
GNU Affero General Public License v3.0
4.17k stars 535 forks source link

Mimir OS Enabling Tenant Federation: true in mimir store gateway config is breaking the store component. #9986

Closed prab2014 closed 3 days ago

prab2014 commented 3 days ago

What is the bug?

Trying to achieve federation as described by the official Grafana mimir channels,

Source : - https://grafana.com/docs/mimir/latest/manage/secure/authentication-and-authorization/

You can federate queries across multiple tenants by using true in -tenant-federation.enabled=true. When you specify tenant IDs, separate them with a pipe (|) character in the X-Scope-OrgID header, as in the example X-Scope-OrgID: tenant-1|tenant-2|tenant-3.

Trying to set the tenant -federation field to True for mimir store gateway component but receiving the below error

"field tenant_federation not found in type storegateway.Config"

The purpose is to fetch the query results from the multiple tenants . Bigger question , is tenant federation is not part of mimir OS version? only GEM can have federation?

How to reproduce it?

Deployed latest mimir using helm chart on Kubernetes . To test the tenant federation, currently, setting the tenant-federation value as true in the config map directly.

store_gateway: **#tenant_federation:

enabled: true**

  sharding_ring:
    heartbeat_period: 1m

Once restarted the config map, store gateway pods are crashing along with the ruler components.

What did you think would happen?

Setting tenant federation should facilitates the metrics from multiple tenants. As per the mimir official documentation , we should be able to federate our queries to multiple tenants.

store_gateway:

tenant_federation:

If enabled on all services, queries can be federated across multiple

tenants. The tenant IDs involved need to be specified separated by a '|'

character in the 'X-Scope-OrgID' header.

CLI flag: -tenant-federation.enabled

[enabled: | default = false]**

What was your environment?

Azure Kubernetes

Any additional context to share?

No response