search

k8up-io / k8up

Kubernetes and OpenShift Backup Operator
https://k8up.io/
Apache License 2.0
717 stars 66 forks source link

SIGSEGV in k8up pod when defining archive with empty s3 object #985

Open Barsonax opened 5 months ago

Barsonax commented 5 months ago

Description

After starting the k8up pod after a shortwhile the log below is shown.

Additional Context

No response

Logs

2024-06-27T19:37:27Z    INFO    k8up.operator   Starting workers    {"controller": "check.k8up.io", "controllerGroup": "k8up.io", "controllerKind": "Check", "worker count": 1}
2024-06-27T19:37:27Z    INFO    k8up.operator   Starting workers    {"controller": "schedule.k8up.io", "controllerGroup": "k8up.io", "controllerKind": "Schedule", "worker count": 1}
2024-06-27T19:37:27Z    INFO    k8up.operator   Starting workers    {"controller": "restore.k8up.io", "controllerGroup": "k8up.io", "controllerKind": "Restore", "worker count": 1}
2024-06-27T19:37:27Z    INFO    k8up.operator   Starting workers    {"controller": "archive.k8up.io", "controllerGroup": "k8up.io", "controllerKind": "Archive", "worker count": 1}
2024-06-27T19:37:27Z    INFO    k8up.operator   Starting workers    {"controller": "backup.k8up.io", "controllerGroup": "k8up.io", "controllerKind": "Backup", "worker count": 1}
2024-06-27T19:37:27Z    INFO    k8up.operator   Starting workers    {"controller": "prune.k8up.io", "controllerGroup": "k8up.io", "controllerKind": "Prune", "worker count": 1}
2024-06-27T19:37:27Z    INFO    k8up.operator   Observed a panic in reconciler: runtime error: invalid memory address or nil pointer dereference    {"controller": "archive.k8up.io", "controllerGroup": "k8up.io", "controllerKind": "Archive", "Archive": {"name":"backup-schedule-archive-hnszl","namespace":"homelab"}, "namespace": "homelab", "name": "backup-schedule-archive-hnszl", "reconcileID": "5332a90b-cf9b-4c5b-93ad-439ca6889453"}
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
    panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x13c146c]

goroutine 245 [running]:
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile.func1()
    /home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.2/pkg/internal/controller/controller.go:116 +0x1e5
panic({0x17ca500?, 0x295caa0?})
    /opt/hostedtoolcache/go/1.21.9/x64/src/runtime/panic.go:914 +0x21f
github.com/k8up-io/k8up/v2/api/v1.(*S3Spec).RestoreEnvVars(0x0)
    /home/runner/work/k8up/k8up/api/v1/backend.go:150 +0x2c
github.com/k8up-io/k8up/v2/operator/archivecontroller.(*ArchiveExecutor).setupEnvVars(0xc000571868, {0x1ca7600?, 0xc0008feb40?}, 0xc0001d1b80)
    /home/runner/work/k8up/k8up/operator/archivecontroller/executor.go:99 +0xdf
github.com/k8up-io/k8up/v2/operator/archivecontroller.(*ArchiveExecutor).Execute.func1()
    /home/runner/work/k8up/k8up/operator/archivecontroller/executor.go:56 +0xce
sigs.k8s.io/controller-runtime/pkg/controller/controllerutil.mutate(0x1c8f3a0?, {{0xc000170030?, 0x0?}, {0xc0008efa40?, 0x1ca7600?}}, {0x1cbb7d8, 0xc00094ca00})
    /home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.2/pkg/controller/controllerutil/controllerutil.go:426 +0x49
sigs.k8s.io/controller-runtime/pkg/controller/controllerutil.CreateOrUpdate({0x1ca7600, 0xc0008feb40}, {0x1cafe80, 0xc0001da5a0}, {0x1cbb7d8?, 0xc00094ca00}, 0x0?)
    /home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.2/pkg/controller/controllerutil/controllerutil.go:282 +0x130
github.com/k8up-io/k8up/v2/operator/archivecontroller.(*ArchiveExecutor).Execute(0xc000571868, {0x1ca7600?, 0xc0008feb40})
    /home/runner/work/k8up/k8up/operator/archivecontroller/executor.go:50 +0x18b
github.com/k8up-io/k8up/v2/operator/locker.(*lockerImpl).TryRun(0xc0008febd0, {0x1ca7600, 0xc0008feb40}, {{0x1cafe80, 0xc0001da5a0}, {0x1cbda58, 0xc0001d1b80}, {0xc0008ef980, 0x27}}, 0x0, ...)
    /home/runner/work/k8up/k8up/operator/locker/locker.go:66 +0x652
github.com/k8up-io/k8up/v2/operator/archivecontroller.(*ArchiveReconciler).Provision(0xc0000160e0, {0x1ca7600, 0xc0008feb40}, 0xc0001d1b80)
    /home/runner/work/k8up/k8up/operator/archivecontroller/controller.go:61 +0x6ca
github.com/k8up-io/k8up/v2/operator/reconciler.(*controller[...]).Reconcile(0x1c8e9a0, {0x1ca7600, 0xc0008feb40}, {{{0xc000170030, 0x0?}, {0xc000044300?, 0x5?}}})
    /home/runner/work/k8up/k8up/operator/reconciler/reconciler.go:57 +0x1eb
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0x1caa8a8?, {0x1ca7600?, 0xc0008feb40?}, {{{0xc000170030?, 0xb?}, {0xc000044300?, 0x0?}}})
    /home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.2/pkg/internal/controller/controller.go:119 +0xb7
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc000385ea0, {0x1ca7638, 0xc0004fd310}, {0x184fbc0?, 0xc0005a83a0?})
    /home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.2/pkg/internal/controller/controller.go:316 +0x3cc
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc000385ea0, {0x1ca7638, 0xc0004fd310})
    /home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.2/pkg/internal/controller/controller.go:266 +0x1af
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2()
    /home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.2/pkg/internal/controller/controller.go:227 +0x79
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2 in goroutine 92
    /home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.2/pkg/internal/controller/controller.go:223 +0x565

Expected Behavior

No SIGSEGV and working archives or atleast a descriptive error telling me what iam doing wrong.

Steps To Reproduce

Not sure first it ran without problems then suddenly started showing this issue. If I find the cause I wil update this.

EDIT: this config seems to cause it:

  archive:
    schedule: '* * * * *'
    restoreMethod:
     s3:

This however does work:

  archive:
    schedule: '* * * * *'
    restoreMethod:
     s3: {}

Version of K8up

2.10.0

Version of Kubernetes

1.29.5

Distribution of Kubernetes

k3s