movetokube / postgres-operator

Postgres operator for Kubernetes
MIT License
167 stars 59 forks source link

Postgres User reconciller error loop when DB deleted manually - still having this problem in 1.2.1 #115

Open ctaymor opened 1 year ago

ctaymor commented 1 year ago

Version: ghcr.io/movetokube/postgres-operator:1.2.1

Problem:

I manually created a database and then used the postgres User custom resource to create a role for it. I then deleted the database manually. The operator is stuck in a loop of Reconciller errors. Previously, it was crashing as well, however, I updated to 1.2.1 and it is no longer actually crashing. (It stays up, and can reconcille other requests). However, it is constantly trying to reconcile this error, even though it is clear it will not be able to. (It's been trying for weeks now).

Expected Behavior: Either this should be handled gracefully, or there should be some way to manually cancel the reconcille loop, and manually delete the user.

What I've tried:

Stacktrace:

{"level":"info","ts":1689266019.4010785,"logger":"controller_postgresuser","msg":"Reconciling PostgresUser","Request.Namespace":"mastodon","Request.Name":"my-db-test"}
{"level":"error","ts":1689266019.4115453,"logger":"controller-runtime.controller","msg":"Reconciler error","controller":"postgresuser-controller","request":"mastodon/my-db-test","error":"Internal error occurred: Postgres.db.movetokube.com \"my-db-test\" not found","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/src/github.com/movetokube/postgres-operator/vendor/github.com/go-logr/zapr/zapr.go:128\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/src/github.com/movetokube/postgres-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:258\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/src/github.com/movetokube/postgres-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:232\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker\n\t/go/src/github.com/movetokube/postgres-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:211\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/src/github.com/movetokube/postgres-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/src/github.com/movetokube/postgres-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/src/github.com/movetokube/postgres-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/go/src/github.com/movetokube/postgres-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:90"}
{"level":"info","ts":1689266076.696294,"logger":"controller_postgresuser","msg":"Reconciling PostgresUser","Request.Namespace":"my-db-testing","Request.Name":"my-db-test"}

Is there a way to manually cancel the reconcillation loop? Deleting the postgres user resource just makes the operator try to cancel the resource.

Thank you.