grafana / helm-charts

Apache License 2.0
1.67k stars 2.28k forks source link

[Helm] tempo Ingester rollout-group collides with other LTGM component ingesters #3442

Open boldfield opened 14 hours ago

boldfield commented 14 hours ago

Describe the bug

The tempo Helm chart with deploymentMode: Distributed and zoneAwareReplication enabled (default) generates Ingester StatefulSets with labels for rollout-operator. Ex here:

{{- if and .ctx.Values.ingester.zoneAwareReplication.enabled .rolloutZoneName }}
name: {{ printf "%s-%s" .component .rolloutZoneName }}
rollout-group: {{ .component }}
zone: {{ .rolloutZoneName }}
{{- end }}

This is identical to the issue filed against loki, here: https://github.com/grafana/loki/issues/13168

Additionally, the anti-affinity rules for loki/tempo/mimir all incorporate the StatefulSet name. In all three cases it is explicitly set to the component name coupled with the target zone. When deploying multiple LTGM components to the same namespace this yields a first scheduled wins problem.