AppsFlyer / local-pvc-releaser

A Kubernetes controller designed to oversee Persistent Volume Claims (PVCs) associated with local storage on worker nodes. Its purpose is to enhance resilience and facilitate automatic recovery in the event of node termination.
Apache License 2.0
11 stars 0 forks source link

on prem k8s issues #48

Open shemreader opened 1 month ago

shemreader commented 1 month ago

Hi

I have tried to use it on local k8s running on premise and received the following error. {"level":"info","ts":"2024-07-14T10:58:49Z","msg":"Starting workers","dryrun":"false","controller":"event","controllerGroup":"","controllerKind":"Event","worker count":1} {"level":"info","ts":"2024-07-14T10:58:49Z","msg":"node termination event found","dryrun":"false","Message":"Node data-ditestcluster-tf-k8s-worker-1 event: Removing Node data-ditestcluster-tf-k8s-worker-1 from Controller","EventID":"2e89f14c-f572-4b4a-b8f9-725366e94c38","EventTime":"2024-07-14 10:32:30 +0000 UTC"} {"level":"info","ts":"2024-07-14T10:58:49Z","msg":"Observed a panic in reconciler: runtime error: invalid memory address or nil pointer dereference","dryrun":"false","controller":"event","controllerGroup":"","controllerKind":"Event","Event":{"name":"data-ditestcluster-tf-k8s-worker-1.17e20dc5b6f7f23f","namespace":"default"},"namespace":"default","name":"data-ditestcluster-tf-k8s-worker-1.17e20dc5b6f7f23f","reconcileID":"207cfae6-7e58-4f2b-bdee-4f571508ba14"} panic: runtime error: invalid memory address or nil pointer dereference [recovered] panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x140a7ba]

goroutine 110 [running]: sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Reconcile.func1() /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:119 +0x1fa panic({0x157caa0, 0x249cb70}) /usr/local/go/src/runtime/panic.go:884 +0x212 github.com/AppsFlyer/local-pvc-releaser/internal/controller.(PVCReconciler).FilterPVListByNodeName(0xc000230850, 0xc0001ca3f0, {0xc00067c300, 0x22}) /workspace/internal/controller/pvc_controller.go:137 +0xba github.com/AppsFlyer/local-pvc-releaser/internal/controller.(PVCReconciler).Reconcile(0xc000230850, {0x19a1f98, 0xc00057c630}, {{{0xc00067a397?, 0x10?}, {0xc000641880?, 0x40da67?}}}) /workspace/internal/controller/pvc_controller.go:81 +0x374 sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Reconcile(0x19a1f98?, {0x19a1f98?, 0xc00057c630?}, {{{0xc00067a397?, 0x1506820?}, {0xc000641880?, 0x0?}}}) /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:122 +0xc8 sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).reconcileHandler(0xc0001a63c0, {0x19a1ef0, 0xc000700180}, {0x15ceee0?, 0xc000598060?}) /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:323 +0x38f sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).processNextWorkItem(0xc0001a63c0, {0x19a1ef0, 0xc000700180}) /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:274 +0x1d9 sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Start.func2.2() /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:235 +0x85 created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Start.func2 /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:231 +0x333 Logs from 7/14/2024, 1:56:20 PM

PV definition

apiVersion: v1 kind: PersistentVolume metadata: name: pvc-e3f4ba59-f5e5-4a2a-b91b-4263f2b3966f uid: ed6b281c-f3ca-4190-a271-fc299c7a819c resourceVersion: '667358971' creationTimestamp: '2024-07-14T10:25:23Z' annotations: pv.kubernetes.io/provisioned-by: local.csi.openebs.io volume.kubernetes.io/provisioner-deletion-secret-name: '' volume.kubernetes.io/provisioner-deletion-secret-namespace: '' finalizers:

tal-asulin commented 1 month ago

Hi @shemreader, on which Kubernetes version the controller was deployed?

shemreader commented 1 month ago

v1.27.9

On Mon, Jul 15, 2024 at 10:04 AM Tal Asulin @.***> wrote:

Hi @shemreader https://github.com/shemreader, on which Kubernetes version the controller was deployed?

— Reply to this email directly, view it on GitHub https://github.com/AppsFlyer/local-pvc-releaser/issues/48#issuecomment-2227824167, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHW564VGUGTXMWBUBG5KQYLZMNYBNAVCNFSM6AAAAABK35XSJKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRXHAZDIMJWG4 . You are receiving this because you were mentioned.Message ID: @.***>