logging operator keeps restarting after upgrade to 4.5.6

xinity commented 5 months ago

Bugs should be filed for issues encountered whilst operating logging-operator. You should first attempt to resolve your issues through the community support channels, e.g. Slack, in order to rule out individual configuration errors. #logging-operator Please provide as much detail as possible.

Describe the bug: i've updated my logging operator to 4.5.6 using official helm charts and now it keeps restarting with a error saying: {"level":"error","ts":"2024-03-15T11:29:26Z","logger":"controller-runtime.source.EventHandler","msg":"if kind is a CRD, it should be installed before calling Start","kind":"SyslogNGConfig.logging.banzaicloud.io","error":"no matches for kind \"SyslogNGConfig\" in version \"logging.banzaicloud.io/v1beta1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.3/pkg/internal/source/kind.go:63\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func2\n\t/go/pkg/mod/k8s.io/apimachinery@v0.28.4/pkg/util/wait/loop.go:73\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext\n\t/go/pkg/mod/k8s.io/apimachinery@v0.28.4/pkg/util/wait/loop.go:74\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextCancel\n\t/go/pkg/mod/k8s.io/apimachinery@v0.28.4/pkg/util/wait/poll.go:33\nsigs.k8s.io/controller-runtime/pkg/internal/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.3/pkg/internal/source/kind.go:56"}

Expected behaviour: the logging operator should upgrade and run smoothly

Steps to reproduce the bug: deploy the logging-operator using helm version on version 4.4.1 and then upgrade to the latest release

Environment details:

Kubernetes version (e.g. v1.15.2): 1.27
Cloud-provider/provisioner (e.g. AKS, GKE, EKS, PKE etc): GKE
logging-operator version (e.g. 2.1.1): 4.5.6
Install method (e.g. helm or static manifests): helm
Logs from the misbehaving component (and any other relevant logs): see above
Resource definition (possibly in YAML format) that caused the issue, without sensitive data:

/kind bug

pepov commented 5 months ago

@xinity you have to install the new CRDs, which helm doesn't do by default. Here is an example on how to do this manually, but depending on your tool it can vary:

helm show crds oci://ghcr.io/kube-logging/helm-charts/logging-operator --version 4.5.6 | kubectl apply --server-side --force-conflicts -f-

xinity commented 5 months ago

@pepov is it planned to add it into the helm charts or not? (just asking 😉)

pepov commented 5 months ago

@xinity it is indeed part of the helm chart. I meant you have to handle the upgrade of the CRDs for yourself as helm doesn't do it for you automatically: https://helm.sh/docs/chart_best_practices/custom_resource_definitions/#some-caveats-and-explanations

kube-logging / logging-operator

logging operator keeps restarting after upgrade to 4.5.6 #1702