search

kubedb / project

KubeDB Enhancement & Issues
Apache License 2.0
1 stars 0 forks source link

postgres.kubedb.com/postgres-cluster 16.1 is stuck on Provisioning status and never getting ready #2

Open salcsal opened 1 month ago

salcsal commented 1 month ago

I am trying to run postgres ha in AWS eks cluster - everything is ok except pods are not accepting any database connection.

Here is my postgres-cluster.yaml

apiVersion: kubedb.com/v1alpha2 kind: Postgres metadata: name: postgres-cluster namespace: demo spec: version: "16.1" replicas: 3 standbyMode: Hot storageType: Durable storage: storageClassName: "efs-sc" accessModes:

salcsal@boss1:~/kube-db$ kubectl get all -n demo NAME READY STATUS RESTARTS AGE pod/postgres-cluster-0 2/2 Running 0 77m pod/postgres-cluster-1 2/2 Running 0 77m pod/postgres-cluster-2 2/2 Running 0 77m

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/postgres-cluster ClusterIP 10.100.61.251 5432/TCP,2379/TCP 77m service/postgres-cluster-pods ClusterIP None 5432/TCP,2380/TCP,2379/TCP 77m service/postgres-cluster-standby ClusterIP 10.100.246.125 5432/TCP 77m

NAME READY AGE statefulset.apps/postgres-cluster 3/3 77m

NAME TYPE VERSION AGE appbinding.appcatalog.appscode.com/postgres-cluster kubedb.com/postgres 16.1 77m

NAME VERSION STATUS AGE postgres.kubedb.com/postgres-cluster 16.1 Provisioning 77m salcsal@boss1:~/kube-db$

salcsal@boss1:~/kube-db$ kubectl get pvc -n demo NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE data-postgres-cluster-0 Bound pvc-78f18cb2-8c24-4348-9dc5-7e55f6b92978 1Gi RWX efs-sc 79m data-postgres-cluster-1 Bound pvc-d7e1ccde-873b-422a-8861-db200f76a374 1Gi RWX efs-sc 79m data-postgres-cluster-2 Bound pvc-04053293-879b-410d-9901-0ef4ed2d7510 1Gi RWX efs-sc 79m salcsal@boss1:~/kube-db$

Describe shows The Postgres: demo/postgres-cluster is not accepting client requests. error: failed to run query: dial tcp 10.100.61.251:5432: connect: connection refused

salcsal@boss1:~/kube-db$ kubectl -n demo describe postgres.kubedb.com/postgres-cluster Name: postgres-cluster Namespace: demo Labels: Annotations: API Version: kubedb.com/v1alpha2 Kind: Postgres Metadata: Creation Timestamp: 2024-08-03T20:31:40Z Finalizers: kubedb.com Generation: 2 Resource Version: 1580050 UID: 23ac328c-3487-48dc-a0d0-daea4d295d9d Spec: Allowed Schemas: Namespaces: From: Same Arbiter: Resources: Limits: Memory: 256Mi Requests: Cpu: 200m Memory: 256Mi Storage: 2Gi Auth Secret: Name: postgres-cluster-auth Auto Ops: Client Auth Mode: md5 Coordinator: Resources: Limits: Memory: 256Mi Requests: Cpu: 200m Memory: 256Mi Security Context: Allow Privilege Escalation: false Capabilities: Drop: ALL Run As Group: 70 Run As Non Root: true Run As User: 70 Seccomp Profile: Type: RuntimeDefault Health Checker: Failure Threshold: 1 Period Seconds: 10 Timeout Seconds: 10 Leader Election: Election Tick: 10 Heartbeat Tick: 1 Maximum Lag Before Failover: 67108864 Period: 300ms Transfer Leadership Interval: 1s Transfer Leadership Timeout: 1m0s Pod Template: Controller: Metadata: Spec: Affinity: Pod Anti Affinity: Preferred During Scheduling Ignored During Execution: Pod Affinity Term: Label Selector: Match Labels: app.kubernetes.io/instance: postgres-cluster app.kubernetes.io/managed-by: kubedb.com app.kubernetes.io/name: postgreses.kubedb.com Namespaces: demo Topology Key: kubernetes.io/hostname Weight: 100 Pod Affinity Term: Label Selector: Match Labels: app.kubernetes.io/instance: postgres-cluster app.kubernetes.io/managed-by: kubedb.com app.kubernetes.io/name: postgreses.kubedb.com Namespaces: demo Topology Key: topology.kubernetes.io/zone Weight: 50 Container Security Context: Allow Privilege Escalation: false Capabilities: Drop: ALL Run As Group: 70 Run As Non Root: true Run As User: 70 Seccomp Profile: Type: RuntimeDefault Init Containers: Name: postgres-init-container Resources: Limits: Memory: 512Mi Requests: Cpu: 200m Memory: 256Mi Security Context: Allow Privilege Escalation: false Capabilities: Drop: ALL Run As Group: 70 Run As Non Root: true Run As User: 70 Seccomp Profile: Type: RuntimeDefault Resources: Limits: Memory: 1Gi Requests: Cpu: 500m Memory: 1Gi Security Context: Fs Group: 70 Run As Group: 70 Run As User: 70 Service Account Name: postgres-cluster Replicas: 3 Replication: Wal Keep Size: 1024 Wal Limit Policy: WALKeepSize Ssl Mode: disable Standby Mode: Hot Storage: Access Modes: ReadWriteMany Resources: Requests: Storage: 1Gi Storage Class Name: efs-sc Storage Type: Durable Termination Policy: WipeOut Version: 16.1 Status: Conditions: Last Transition Time: 2024-08-03T20:31:40Z Message: The KubeDB operator has started the provisioning of Postgres: demo/postgres-cluster Reason: DatabaseProvisioningStartedSuccessfully Status: True Type: ProvisioningStarted Last Transition Time: 2024-08-03T20:31:55Z Message: All desired replicas are ready. Reason: AllReplicasReady Status: True Type: ReplicaReady Last Transition Time: 2024-08-03T20:32:05Z Message: The Postgres: demo/postgres-cluster is not accepting client requests. error: failed to run query: dial tcp 10.100.61.251:5432: connect: connection refused Observed Generation: 2 Reason: DatabaseNotAcceptingConnectionRequest Status: False Type: AcceptingConnection Last Transition Time: 2024-08-03T20:32:05Z Message: The Postgres: demo/postgres-cluster is not ready. Observed Generation: 2 Reason: ReadinessCheckFailed Status: False Type: Ready Observed Generation: 1 Phase: Provisioning Events: salcsal@boss1:~/kube-db$

Any help will be appreciated.

Thanks,

Salil Chattopadhyay salcsal@hotmail.com

souravbiswassanto commented 3 weeks ago

Hello @salcsal, Sorry for the late. Can you give me the log from postgres an pg-coordinator container from all the pods?