mongodb / mongodb-atlas-kubernetes

MongoDB Atlas Kubernetes Operator - Manage your MongoDB Atlas clusters from Kubernetes
http://www.mongodb.com/cloud/atlas
Apache License 2.0
146 stars 75 forks source link

Panic reconciling deployment backup schedule #589

Closed mscdw closed 2 years ago

mscdw commented 2 years ago

What did you do to encounter the bug? Updated backupRef to reference a backup schedule

What did you expect? Reconciliation without panic

What happened instead? { "textPayload": "panic: runtime error: invalid memory address or nil pointer dereference\n[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x13b196d]\n\ngoroutine 354 [running]:\ngithub.com/mongodb/mongodb-atlas-kubernetes/pkg/controller/atlasdeployment.(*AtlasDeploymentReconciler).handleDeploymentBackupSchedule(0xc0003eac00, 0xc000138900, 0xc0000e65a0, {0xc000676750, 0x18}, {0xc00095f940, 0xd}, 0x6, {{{0xc000577cf0, 0xc}, ...}})\n\t/workspace/pkg/controller/atlasdeployment/atlasdeployment_controller.go:268 +0xbcd\ngithub.com/mongodb/mongodb-atlas-kubernetes/pkg/controller/atlasdeployment.(*AtlasDeploymentReconciler).handleRegularDeployment(0xc0004f0240, 0xc000138900, 0xc0006906c0, 0x13a554f, {{{0xc000577cf0, 0xd}, {0xc000577cd0, 0x4}}})\n\t/workspace/pkg/controller/atlasdeployment/atlasdeployment_controller.go:327 +0x365\ngithub.com/mongodb/mongodb-atlas-kubernetes/pkg/controller/atlasdeployment.(*AtlasDeploymentReconciler).Reconcile(0xc0003eac00, {0xc0004dfce0, 0x151b100}, {{{0xc000577cf0, 0xc}, {0xc000577cd0, 0xf}}})\n\t/workspace/pkg/controller/atlasdeployment/atlasdeployment_controller.go:138 +0xb19\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0xc00015c2c0, {0x1900938, 0xc0004dfce0}, {{{0xc000577cf0, 0x162e500}, {0xc000577cd0, 0x4156f4}}})\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.1/pkg/internal/controller/controller.go:114 +0x26f\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc00015c2c0, {0x1900890, 0xc0005ef940}, {0x1578f40, 0xc000145700})\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.1/pkg/internal/controller/controller.go:311 +0x33e\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc00015c2c0, {0x1900890, 0xc0005ef940})\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.1/pkg/internal/controller/controller.go:266 +0x205\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2()\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.1/pkg/internal/controller/controller.go:227 +0x85\ncreated by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.1/pkg/internal/controller/controller.go:223 +0x357", "insertId": "acvk5uet4hgvta49", "resource": { "type": "k8s_container", "labels": { "project_id": "x", "namespace_name": "x", "location": "us-west1-a", "cluster_name": "prometheus", "pod_name": "mongodb-atlas-operator-7f58799456-grxws", "container_name": "manager" } }, "timestamp": "2022-07-04T16:37:08.704112223Z", "severity": "ERROR", "labels": { "k8s-pod/app_kubernetes_io/name": "mongodb-atlas-operator", "k8s-pod/app_kubernetes_io/instance": "atlas-operator", "k8s-pod/pod-template-hash": "7f58799456", "compute.googleapis.com/resource_name": "gke-x-default-pool-9c92b326-0l12" }, "logName": "projects/x/logs/stderr", "receiveTimestamp": "2022-07-04T16:37:27.732397463Z" }

Operator Information

Kubernetes Cluster Information

Additional context Add any other context about the problem here.

If possible, please include:

igor-karpukhin commented 2 years ago

Hi @mattsmith-x. Thanks for creating this issue and attaching your resources! Could you please also provide logs (e.g. 50-100 lines) from the operator just before this panic message? As well as a description of your atlasbackuppolicy resource. Thanks in advance!