SAP / sap-btp-service-operator

SAP BTP service operator enables developers to connect Kubernetes clusters to SAP BTP accounts and to consume SAP BTP services within the clusters by using Kubernetes native tools.
Apache License 2.0
125 stars 49 forks source link

SAP BTP OPERATOR in our k8s cluster is crashing infinitely #464

Open tarun00000 opened 6 days ago

tarun00000 commented 6 days ago

Version: ghcr.io/sap/sap-btp-service-operator/controller:v0.5.4

I tried to make some changes and provision btp service instance and btp service binding.

But the service instance manifest had some issues and caused the BTP operator to crash .

2024-09-09T15:06:49Z INFO Observed a panic in reconciler: runtime error: invalid memory address or nil pointer dereference {"controller": "servicebinding", "controllerGroup": "services.cloud.sap.com", "controllerKind": "ServiceBinding", "ServiceBinding": {"name":"argocd-key","namespace":"default"}, "namespace": "default", "name": "argocd-key", "reconcileID": "6379c665-8827-448e-b162-006030fc2bc2"}

I am currently trying to delete the BTP service binding manually or forcefully, but it is stuck because the deletion process attempts to communicate with the BTP operator, which is in a crashing state And i do not have access to the btp operator namespace so I can not delete btp operator pod for reconciliation.

Logs

2024-09-09T15:06:49Z INFO Observed a panic in reconciler: runtime error: invalid memory address or nil pointer dereference {"controller": "servicebinding", "controllerGroup": "[services.cloud.sap.com](http://services.cloud.sap.com/)", "controllerKind": "ServiceBinding", "ServiceBinding": {"name":"argocd-key","namespace":"default"}, "namespace": "default", "name": "argocd-key", "reconcileID": "6379c665-8827-448e-b162-006030fc2bc2"}

2024-09-09T14:50:46Z INFO Observed a panic in reconciler: runtime error: invalid memory address or nil pointer dereference {"controller": "servicebpanic: runtime error: invalid memory address or nil pointer dereference [recovered] panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x1a0 pc=0x14a06f2] goroutine 134 [running]: sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile.func1() /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:115 +0x1e5 panic({0x1606840?, 0x25f2eb0?}) /usr/local/go/src/runtime/panic.go:914 +0x21f github.com/SAP/sap-btp-service-operator/controllers.(*ServiceBindingReconciler).Reconcile(0xc00049b7f8, {0x1a5a1d8, 0xc000438fc0}, {{{0xc000396920, 0x7}, {0 /workspace/controllers/servicebinding_controller.go:96 +0x5b2 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0x1a5a1d8?, {0x1a5a1d8?, 0xc000438fc0?}, {{{0xc000396920?, 0x155d620?}, {0xc0 /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:118 +0xb7 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc000186140, {0x1a5a210, 0xc000024280}, {0x167c600?, 0xc00041d520?}) /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:314 +0x365 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc000186140, {0x1a5a210, 0xc000024280}) /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:265 +0x1c9 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2() /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:226 +0x79 created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2 in goroutine 102 /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:222 +0x565 Stream closed EOF for sap-btp-service-operator/sap-btp-operator-controller-manager-547cf79474-w7x85 (manager)

I also tried

try to delete the ServiceBinding argocd-key and remove its finalizer. but did not work. error I am getting is

 error: servicebindings.services.cloud.sap.com "argocd-key" could not be patched: Internal error occurred: failed calling webhook "mservicebinding.kb.io": failed to call webhook: Post "https://sap-btp-operator-webhook-service.sap-btp-service-operator.svc:443/mutate-services-cloud-sap-com-v1-servicebinding?timeout=10s": no endpoints available for service "sap-btp-operator-webhook-service"
You can run `kubectl replace -f /var/folders/24/pdjyg77x30gdnd70pcm2z7jw0000gn/T/kubectl-edit-3513063779.yaml` to try this update again.

Need your help, Thanks

kerenlahav commented 6 days ago

Hi @tarun00000 This issue has been resolved in v0.5.10, you are using a very old version of the operator. please upgrade the operator and try again.

tarun00000 commented 5 days ago

Hi kerenlahav, Thankyou for the response. I will test and close this issue.