Controller crash with panic: runtime error: invalid memory address or nil pointer dereference

[greenmaid]

Hello,

using the provided CR example https://github.com/seaweedfs/seaweedfs-operator/blob/master/config/samples/seaweed_v1_seaweed.yaml, controller is crashing with following error:

1.7089442396660287e+09  INFO    controllers.Seaweed     pod     {"sw-master-statefulset": "seaweed1", "name": "seaweed1-master-2", "containerStatus": {"name":"master","state":{"running":{"startedAt":"2024-02-26T10:38:39Z"}},"lastState":{"terminated":{"exitCode":255,"reason":"Error","startedAt":"2024-02-26T10:38:20Z","finishedAt":"2024-02-26T10:38:20Z","containerID":"containerd://43c309d639ce1b056e379c1bc3c076f5e43e8729ab55a4d0f0bef234c734df70"}},"ready":true,"restartCount":2,"image":"docker.io/chrislusf/seaweedfs:latest","imageID":"docker.io/chrislusf/seaweedfs@sha256:b756ecbfd268c91a7d8ae4c8414f93ec46a6df274a5f8cc4cf14adaa69f26d85","containerID":"containerd://5bf2bf24941bcf27fbc394c908700070ed09e72b94c06f447dea21aed9752370","started":true}}
1.7089442396660378e+09  INFO    controllers.Seaweed     masters are ready       {"sw-master-statefulset": "seaweed1"}
1.7089442396703095e+09  INFO    controllers.Seaweed     ensure volume peer service seaweed1-volume-peer {"sw-volume-peer-service": "seaweed1"}
1.7089442396820629e+09  INFO    controllers.Seaweed     ensure volume service seaweed1-volume-0 {"sw-volume-service": "seaweed1", "index": 0, "index": 0}
1.7089442396896477e+09  INFO    controllers.Seaweed     ensure volume stateful set seaweed1-volume      {"sw-volume-statefulset": "seaweed1"}
1.7089442396934583e+09  INFO    controllers.Seaweed     ensure filer peer service seaweed1-filer-peer   {"sw-filer-peer-service": "seaweed1"}
1.7089442397012525e+09  INFO    controllers.Seaweed     ensure filer service seaweed1-filer     {"sw-filer-service": "seaweed1"}
1.708944239704081e+09   INFO    controllers.Seaweed     Get filer ConfigMap seaweed1-filer      {"sw-filer-configmap": "seaweed1"}
1.7089442397041237e+09  INFO    Observed a panic in reconciler: runtime error: invalid memory address or nil pointer dereference        {"controller": "seaweed", "controllerGroup": "seaweed.seaweedfs.com", "controllerKind": "Seaweed", "seaweed": {"name":"seaweed1","namespace":"seaweed"}, "namespace": "seaweed", "name": "seaweed1", "reconcileID": "0442909c-1dad-45bc-a442-e5db19fba260"}
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
        panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x197802d]

goroutine 404 [running]:
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile.func1()
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:118 +0x1da
panic({0x1b5d920?, 0x322def0?})
        /usr/local/go/src/runtime/panic.go:914 +0x21f
github.com/seaweedfs/seaweedfs-operator/controllers.(*SeaweedReconciler).createFilerStatefulSet(0xc000592bd0?, 0xc000788000)
        /workspace/controllers/controller_filer_statefulset.go:80 +0x6cd
github.com/seaweedfs/seaweedfs-operator/controllers.(*SeaweedReconciler).ensureFilerStatefulSet(0xc000592c00, 0xc000788000)
        /workspace/controllers/controller_filer.go:48 +0x10d
github.com/seaweedfs/seaweedfs-operator/controllers.(*SeaweedReconciler).ensureFilerServers(0xc000592c00, 0xc000788000)
        /workspace/controllers/controller_filer.go:32 +0xeb
github.com/seaweedfs/seaweedfs-operator/controllers.(*SeaweedReconciler).Reconcile(0xc000592c00, {0x2302348, 0xc0007286f0}, {{{0xc0007ab118, 0x7}, {0xc0007ab110, 0x8}}})
        /workspace/controllers/seaweed_controller.go:67 +0x1df
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0x2302380?, {0x2302348?, 0xc0007286f0?}, {{{0xc0007ab118?, 0x1d26760?}, {0xc0007ab110?, 0xc0000cef40?}}})
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:121 +0xb7
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc00097b680, {0x2302380, 0xc0008618b0}, {0x1be58e0?, 0xc000420c80?})
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:320 +0x2fc
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc00097b680, {0x2302380, 0xc0008618b0})
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:273 +0x1c9
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2()
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:234 +0x79
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2 in goroutine 386
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:230 +0x565

Note: controler is run with image ghcr.io/seaweedfs/seaweedfs-operator:latest

[chrislusf]

I could not locate which version of source code it is.

[greenmaid]

I reproduce it with image built from current main (commit f205ad4)

[greenmaid]

but image from previous main (commit bb4ec87) is working fine

[chrislusf]

Thanks for the git commit hash.

The line controllers/controller_filer_statefulset.go:80 is reading m.Spec.Filer.Persistence.Enabled. There should be a nil check for m.Spec.Filer.Persistence.

    if m.Spec.Filer.Persistence != nil && m.Spec.Filer.Persistence.Enabled {

Could you please try it? Send a PR if it works?

[greenmaid]

it works for me. PR proposed