can't upgrade loki

[nareto]

Premise: this is most probably user error - I would've opened a discussion instead of an issue but it's not enabled on this repo.

I originally deployed Loki with:

helm upgrade --install loki grafana/loki \
    --namespace prometheus \
    --set loki.persistence.enabled=true \
    --set loki.persistence.storageClassName="gp2" \
    --set loki.persistence.size=5Gi

Everything works fine but it's running out of disk space, so I would like to increase the persistence size if possible, if not just replace it with a larger one.

I tried running

helm upgrade --namespace prometheus --set loki.persistence.size=15Gi loki grafana/loki 

but I get this error:

W0504 15:46:06.763527     316 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
W0504 15:46:06.823555     316 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
W0504 15:46:06.896958     316 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
Error: UPGRADE FAILED: cannot patch "loki" with kind StatefulSet: StatefulSet.apps "loki" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', and 'updateStrategy' are forbidden

What am I doing wrong? I get the same error even if I remove the --set option.

[mkenne11]

I'm also getting the same error when trying to upgrade Loki to 2.5.0.

[JWilson45]

I found Loki to have a lot of issues when using local Kubernetes storage. I would recommend moving to object storage or some other supported external storage. It will save you from more problems in the future.

[trevorwhitney]

I think you need to re-create the stateful set for the change you are trying to make there. I also +1 to using object storage.

[mkhodave]

I also facing same issue, I recommend to add this feature for increasing persistence storage on the fly