zalando / postgres-operator

Postgres operator creates and manages PostgreSQL clusters running in Kubernetes
https://postgres-operator.readthedocs.io/
MIT License
4.38k stars 980 forks source link

When the operator is stopped, deleting the PostgreSQL CR, and then restarting the operator, the related resources (such as sts, service, ep, etc.) are not deleted. #2678

Closed fm2022aa closed 2 months ago

fm2022aa commented 5 months ago

Please, answer some short questions which should help us to understand your problem / question better?

operator log:

INFO[0004] listening on :8080                            pkg=apiserver
INFO[0004] ADD event has been queued                     cluster-name=default/acid-minimal-cluster-7 pkg=controller worker=0
INFO[0004] syncing of the cluster started                cluster-name=default/acid-minimal-cluster-7 pkg=controller worker=0
INFO[0004] cluster has a DeletionTimestamp of 2024-06-28T11:55:28+08:00, starting deletion now.  cluster-name=default/acid-minimal-cluster-7 pkg=controller worker=0
DEBU[0004] new node has been added: /openpie-control-plane (kind://docker/openpie/openpie-control-plane)  pkg=controller
DEBU[0004] new node has been added: /openpie-worker (kind://docker/openpie/openpie-worker)  pkg=controller
DEBU[0004] new node has been added: /openpie-worker2 (kind://docker/openpie/openpie-worker2)  pkg=controller
DEBU[0004] new node has been added: /openpie-worker3 (kind://docker/openpie/openpie-worker3)  pkg=controller
INFO[0004] removing the logical backup job               cluster-name=default/acid-minimal-cluster-7 pkg=cluster
DEBU[0004] logical backup cron job "logical-backup-acid-minimal-cluster-7" has already been deleted  cluster-name=default/acid-minimal-cluster-7 pkg=cluster
DEBU[0004] deleting statefulset                          cluster-name=default/acid-minimal-cluster-7 pkg=cluster
DEBU[0004] there is no statefulset in the cluster        cluster-name=default/acid-minimal-cluster-7 pkg=cluster
DEBU[0004] deleting pod disruption budget                cluster-name=default/acid-minimal-cluster-7 pkg=cluster
DEBU[0004] there is no pod disruption budget in the cluster  cluster-name=default/acid-minimal-cluster-7 pkg=cluster
DEBU[0004] deleting endpoint                             cluster-name=default/acid-minimal-cluster-7 pkg=cluster
DEBU[0004] there is no master endpoint in the cluster    cluster-name=default/acid-minimal-cluster-7 pkg=cluster
DEBU[0004] deleting service master                       cluster-name=default/acid-minimal-cluster-7 pkg=cluster
DEBU[0004] No service for master role was found, nothing to delete  cluster-name=default/acid-minimal-cluster-7 pkg=cluster
DEBU[0004] deleting endpoint                             cluster-name=default/acid-minimal-cluster-7 pkg=cluster
DEBU[0004] there is no replica endpoint in the cluster   cluster-name=default/acid-minimal-cluster-7 pkg=cluster
DEBU[0004] deleting service replica                      cluster-name=default/acid-minimal-cluster-7 pkg=cluster
DEBU[0004] No service for replica role was found, nothing to delete  cluster-name=default/acid-minimal-cluster-7 pkg=cluster
DEBU[0004] removing leftover Patroni objects (endpoints / services and configmaps)  cluster-name=default/acid-minimal-cluster-7 pkg=cluster
DEBU[0004] deleting service "default/acid-minimal-cluster-7-config"  cluster-name=default/acid-minimal-cluster-7 pkg=cluster
INFO[0004] deleting connection pooler spilo-role=master  cluster-name=default/acid-minimal-cluster-7 pkg=cluster
DEBU[0004] no connection pooler to delete                cluster-name=default/acid-minimal-cluster-7 pkg=cluster
INFO[0004] deleting connection pooler spilo-role=replica  cluster-name=default/acid-minimal-cluster-7 pkg=cluster
DEBU[0004] no connection pooler to delete                cluster-name=default/acid-minimal-cluster-7 pkg=cluster
INFO[0004] removing finalizer postgres-operator.acid.zalan.do  cluster-name=default/acid-minimal-cluster-7 pkg=cluster
INFO[0004] DELETE event has been queued                  cluster-name=default/acid-minimal-cluster-7 pkg=controller worker=0
INFO[0004] received add event for already existing Postgres cluster  cluster-name=default/acid-minimal-cluster-7 pkg=controller worker=0
INFO[0004] cluster has been deleted                      cluster-name=default/acid-minimal-cluster-7 pkg=controlle

It is possible that the statefulset data is not synchronized when the operator process is up, as indicated by the message "there is no statefulset in the cluster".

fm2022aa commented 4 months ago

enable the finalizer

FxKu commented 3 months ago

Sounds like you are waiting for support for OwnerReferences to arrive with #2688

fm2022aa commented 3 months ago

Sounds like you are waiting for support for OwnerReferences to arrive with #2688

thanks, another problem, when I restart operator, then create a CR. it hanged here.: kubectl logs -f postgres-operator-569c9ff866-fkmfr

image image

After 10 minutes, it get to be "Running".

image

When did it take 10 minutes?

image

using the latest version operator

fm2022aa commented 2 months ago

control by the config"pod_label_wait_timeout" and "resource_check_timeout"