search

grafana / loki

Like Prometheus, but for logs.
https://grafana.com/loki
GNU Affero General Public License v3.0
22.77k stars 3.31k forks source link

loki ingester not starting: flag not defined: -ingester.ring.instance-availability-zone #13061

Open kirankumar-grootan opened 1 month ago

kirankumar-grootan commented 1 month ago

when installing loki 6.6.2 helm chart in distributed mode. ingesters are not starting it says https://github.com/grafana/loki/blob/main/production/helm/loki/templates/ingester/statefulset-ingester.yaml#L90

loki flag provided but not defined: -ingester.ring.instance-availability-zone
Run with -help to get list of available parameters

as per doc i dont find any argument

this version of chart is installing loki version 3.0.0

pls make pod anti affinity as empty in chart by default or its good to add app.kubernetes.io/name: loki label to affinity selector as it is getting conflicted with other ingester (mimir, tempo etc) and pod not scheduling

Arno-Pretorius commented 1 month ago

I am getting the same error as above and I am using Loki with Helm Chart 6.6.0 using Loki version 3.0.0

kirankumar-grootan commented 1 month ago

yes! as per the Loki doc this argument is available for other Loki components but not for Ingester. even Mimir has this argument documented under Ingester and few other Mimir components

Arno-Pretorius commented 1 month ago

I created this PR because if I remove the parament from my live manifest my Ingester deploys and functions normally. https://github.com/grafana/loki/pull/13118

lindeskar commented 1 month ago

Shouldn't it be ingester.availability-zone like in the zone-aware StatefulSets?

https://github.com/grafana/loki/blob/30df31e28b5c360ffed2dea3b47f515e4e24146d/production/helm/loki/templates/ingester/statefulset-ingester-zone-a.yaml#L103

It's also documented in: https://grafana.com/docs/loki/latest/configure/#ingester

Arno-Pretorius commented 1 month ago

Shouldn't it be ingester.availability-zone like in the zone-aware StatefulSets?

https://github.com/grafana/loki/blob/30df31e28b5c360ffed2dea3b47f515e4e24146d/production/helm/loki/templates/ingester/statefulset-ingester-zone-a.yaml#L103

It's also documented in: https://grafana.com/docs/loki/latest/configure/#ingester

Dis changes works I made the change in my Live manifest and the pods are running fine now.

kirankumar-grootan commented 2 weeks ago

@Arno-Pretorius @lindeskar any ETA when this PR https://github.com/grafana/loki/pull/13118 will be merged?

lindeskar commented 2 weeks ago

@kirankumar-grootan I also want it fixed, but I'm not a maintainer. 🤷🏻

rlees85 commented 2 weeks ago

@Arno-Pretorius @lindeskar any ETA when this PR #13118 will be merged?

When one of the two contributors who have submitted PRs (thanks both btw) implement the requested changes ... Hopefully soon or I'll have to submit a third.

I'm not a maintainer by the way, this isn't an authoritative response but its what I assume is holding it up

Arno-Pretorius commented 1 week ago

@lindeskar and @bulmust I updated the PR with the new changes and Reset the review just waiting for approval the argument is now set to "-ingester.availability-zone=zone-default"