search

vitabaks / postgresql_cluster

PostgreSQL High-Availability Cluster (based on "Patroni" and DCS "etcd" or "consul"). Automating with Ansible.
MIT License
1.29k stars 352 forks source link

HaProxy problems and node is not working #149

Closed mcankck closed 2 years ago

mcankck commented 2 years ago

Hi, I have a problem this problem my postgresql cluster's node is not working. I checking cluster log.

log is bellow;

[root@PDDCLPSPSQL03SU log]# patronictl topology
+-------------------+---------------+---------+---------+----+-----------+
| Member            | Host          | Role    | State   | TL | Lag in MB |
+ Cluster: pudo-postgres-cluster (7055798345681484839) -+----+-----------+
| PDDCLPSPSQL03SU   | x.x.x.x | Leader  | running |  2 |           |
| + PDDCLPSPSQL04SU | x.x.x.x | Replica | running |  2 |         0 |
+-------------------+---------------+---------+---------+----+-----------+
[root@PDDCLPSPSQL02SU ~]# journalctl -xe
Feb 28 12:50:17 PDDCLPSPSQL02SU haproxy[1037]: Proxy replicas_sync started.
Feb 28 12:50:17 PDDCLPSPSQL02SU haproxy[1037]: Proxy replicas_sync started.
Feb 28 12:50:17 PDDCLPSPSQL02SU haproxy[1037]: Proxy replicas_async started.
Feb 28 12:50:17 PDDCLPSPSQL02SU haproxy[1037]: Proxy replicas_async started.
Feb 28 12:50:17 PDDCLPSPSQL02SU haproxy[1037]: [WARNING] 058/125017 (1037) : Former worker 6774 exited with code 0
Feb 28 12:50:17 PDDCLPSPSQL02SU confd[1902]: 2022-02-28T12:50:17+03:00 PDDCLPSPSQL02SU /usr/local/bin/confd[1902]: INFO /etc/haproxy/haproxy.cfg has md5sum f5e3ac62b65d9ef8726680c0117fd8a4 should be c7ad0ebd4>
Feb 28 12:50:17 PDDCLPSPSQL02SU confd[1902]: 2022-02-28T12:50:17+03:00 PDDCLPSPSQL02SU /usr/local/bin/confd[1902]: INFO Target config /etc/haproxy/haproxy.cfg out of sync
Feb 28 12:50:17 PDDCLPSPSQL02SU systemd[1]: Reloading HAProxy Load Balancer.
-- Subject: Unit haproxy.service has begun reloading its configuration
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- Unit haproxy.service has begun reloading its configuration
Feb 28 12:50:17 PDDCLPSPSQL02SU haproxy[1037]: [WARNING] 058/125017 (1037) : Reexecuting Master process
Feb 28 12:50:17 PDDCLPSPSQL02SU haproxy[1037]: Proxy stats started.
Feb 28 12:50:17 PDDCLPSPSQL02SU haproxy[1037]: Proxy stats started.
Feb 28 12:50:17 PDDCLPSPSQL02SU haproxy[1037]: Proxy master started.
Feb 28 12:50:17 PDDCLPSPSQL02SU haproxy[1037]: Proxy master started.
Feb 28 12:50:17 PDDCLPSPSQL02SU haproxy[1037]: Proxy replicas started.
Feb 28 12:50:17 PDDCLPSPSQL02SU haproxy[1037]: Proxy replicas started.
Feb 28 12:50:17 PDDCLPSPSQL02SU haproxy[1037]: Proxy replicas_sync started.
Feb 28 12:50:17 PDDCLPSPSQL02SU haproxy[1037]: Proxy replicas_sync started.
Feb 28 12:50:17 PDDCLPSPSQL02SU haproxy[1037]: Proxy replicas_async started.
Feb 28 12:50:17 PDDCLPSPSQL02SU haproxy[1037]: Proxy replicas_async started.
Feb 28 12:50:17 PDDCLPSPSQL02SU systemd[1]: Reloaded HAProxy Load Balancer.
-- Subject: Unit haproxy.service has finished reloading its configuration
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- Unit haproxy.service has finished reloading its configuration
--
-- The result is done.
Feb 28 12:50:17 PDDCLPSPSQL02SU confd[1902]: 2022-02-28T12:50:17+03:00 PDDCLPSPSQL02SU /usr/local/bin/confd[1902]: INFO Target config /etc/haproxy/haproxy.cfg has been updated
Feb 28 12:50:17 PDDCLPSPSQL02SU haproxy[1037]: [WARNING] 058/125017 (1037) : Former worker 6812 exited with code 0
Feb 28 12:50:19 PDDCLPSPSQL02SU systemd[1]: Starting PostgreSQL 14 database server...
-- Subject: Unit postgresql-14.service has begun start-up
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- Unit postgresql-14.service has begun starting up.
Feb 28 12:50:19 PDDCLPSPSQL02SU postmaster[6838]: 2022-02-28 12:50:19 +03 [6838-1]  FATAL:  lock file "postmaster.pid" already exists
Feb 28 12:50:19 PDDCLPSPSQL02SU postmaster[6838]: 2022-02-28 12:50:19 +03 [6838-2]  HINT:  Is another postmaster (PID 1866) running in data directory "/var/lib/pgsql/14/data"?
Feb 28 12:50:19 PDDCLPSPSQL02SU systemd[1]: postgresql-14.service: Main process exited, code=exited, status=1/FAILURE
Feb 28 12:50:19 PDDCLPSPSQL02SU systemd[1]: postgresql-14.service: Failed with result 'exit-code'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- The unit postgresql-14.service has entered the 'failed' state with result 'exit-code'.
Feb 28 12:50:19 PDDCLPSPSQL02SU systemd[1]: Failed to start PostgreSQL 14 database server.
-- Subject: Unit postgresql-14.service has failed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- Unit postgresql-14.service has failed.
--
-- The result is failed.
 server PDDCLPSPSQL03SU x.x.x.x:6432 check port 8008
 server PDDCLPSPSQL04SU x.x.x.x:6432 check port 8008

When I look at haproxy conf, the machine named PDDCLPSPSQL02SU does not appear.

vitabaks commented 2 years ago

Hi @mcankck

Please attach the Patroni logs from PDDCLPSPSQL02SU

sudo journalctl -u patroni

And postgres logs.

mcankck commented 2 years ago

of course log is bellow;

[root@PDDCLPSPSQL02SU ~]# sudo journalctl -u patroni
-- Logs begin at Mon 2022-02-28 12:33:28 +03, end at Mon 2022-02-28 15:33:26 +03. --
Feb 28 15:33:21 PDDCLPSPSQL02SU systemd[1]: Started Runners to orchestrate a high-availability PostgreSQL - patroni.
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 15:33:23,178 INFO: Selected new etcd server http://10.255.117.70:2379
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 15:33:23,187 INFO: No PostgreSQL configuration items changed, nothing to reload.
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 15:33:23,377 WARNING: Postgresql is not running.
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 15:33:23,377 INFO: Lock owner: PDDCLPSPSQL03SU; I am PDDCLPSPSQL02SU
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 15:33:23,395 INFO: pg_controldata:
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   pg_control version number: 1300
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Catalog version number: 202107181
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Database system identifier: 7055798345681484839
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Database cluster state: shut down in recovery
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   pg_control last modified: Mon Feb 28 12:33:03 2022
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Latest checkpoint location: 0/9002098
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Latest checkpoint's REDO location: 0/9002060
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Latest checkpoint's REDO WAL file: 000000020000000000000009
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Latest checkpoint's TimeLineID: 2
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Latest checkpoint's PrevTimeLineID: 2
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Latest checkpoint's full_page_writes: on
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Latest checkpoint's NextXID: 0:743
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Latest checkpoint's NextOID: 24579
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Latest checkpoint's NextMultiXactId: 1
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Latest checkpoint's NextMultiOffset: 0
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Latest checkpoint's oldestXID: 727
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Latest checkpoint's oldestXID's DB: 1
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Latest checkpoint's oldestActiveXID: 743
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Latest checkpoint's oldestMultiXid: 1
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Latest checkpoint's oldestMulti's DB: 1
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Latest checkpoint's oldestCommitTsXid: 0
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Latest checkpoint's newestCommitTsXid: 0
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Time of latest checkpoint: Tue Feb  8 16:42:00 2022
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Fake LSN counter for unlogged rels: 0/3E8
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Minimum recovery ending location: 0/A000000
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Min recovery ending loc's timeline: 2
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Backup start location: 0/0
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Backup end location: 0/0
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   End-of-backup record required: no
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   wal_level setting: replica
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   wal_log_hints setting: on
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   max_connections setting: 1000
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   max_worker_processes setting: 8
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   max_wal_senders setting: 10
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   max_prepared_xacts setting: 0
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   max_locks_per_xact setting: 64
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   track_commit_timestamp setting: off
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Maximum data alignment: 8
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Database block size: 8192
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Blocks per segment of large relation: 131072
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   WAL block size: 8192
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Bytes per WAL segment: 16777216
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Maximum length of identifiers: 64
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Maximum columns in an index: 32
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Maximum size of a TOAST chunk: 1996
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Size of a large-object chunk: 2048
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Date/time type storage: 64-bit integers
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Float8 argument passing: by value
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Data page checksum version: 1
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]:   Mock authentication nonce: 2c987c6384ecbb0fdb5d7f0d9ecbb8e69d83e5ac5ca86add7a68b6f654082323
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 15:33:23,413 INFO: Lock owner: PDDCLPSPSQL03SU; I am PDDCLPSPSQL02SU
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 15:33:23,479 INFO: Local timeline=2 lsn=0/A000000
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 15:33:23,823 INFO: master_timeline=2
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 15:33:23,824 INFO: Lock owner: PDDCLPSPSQL03SU; I am PDDCLPSPSQL02SU
Feb 28 15:33:23 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 15:33:23,862 INFO: starting as a secondary
Feb 28 15:33:24 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 15:33:24,323 INFO: postmaster pid=1866
Feb 28 15:33:24 PDDCLPSPSQL02SU patroni[987]: /var/run/postgresql:5432 - no response
Feb 28 15:33:24 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 15:33:24 +03 [1866-1]  LOG:  redirecting log output to logging collector process
Feb 28 15:33:24 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 15:33:24 +03 [1866-2]  HINT:  Future log output will appear in directory "/var/log/postgresql".
Feb 28 15:33:25 PDDCLPSPSQL02SU patroni[987]: /var/run/postgresql:5432 - accepting connections
Feb 28 15:33:25 PDDCLPSPSQL02SU patroni[987]: /var/run/postgresql:5432 - accepting connections
Feb 28 15:33:25 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 15:33:25,477 INFO: establishing a new patroni connection to the postgres cluster
Feb 28 15:33:25 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 15:33:25,576 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 15:33:26 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 15:33:26,069 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:33:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:33:37,449 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:33:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:33:47,446 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:33:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:33:57,452 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:34:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:34:07,447 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:34:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:34:17,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:34:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:34:27,447 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:34:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:34:37,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:34:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:34:47,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:34:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:34:57,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:35:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:35:07,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:35:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:35:17,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:35:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:35:27,446 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:35:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:35:37,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:35:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:35:47,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:35:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:35:57,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:36:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:36:07,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:36:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:36:17,447 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:36:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:36:27,447 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:36:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:36:37,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:36:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:36:47,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:36:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:36:57,446 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:37:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:37:07,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:37:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:37:17,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:37:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:37:27,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:37:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:37:37,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:37:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:37:47,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:37:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:37:57,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:38:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:38:07,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:38:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:38:17,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:38:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:38:27,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:38:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:38:37,446 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:38:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:38:47,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:38:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:38:57,446 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:39:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:39:07,447 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:39:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:39:17,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:39:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:39:27,446 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:39:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:39:37,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:39:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:39:47,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:39:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:39:57,507 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:40:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:40:07,454 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:40:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:40:17,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:40:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:40:27,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:40:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:40:37,450 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:40:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:40:47,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:40:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:40:57,446 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:41:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:41:07,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:41:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:41:17,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:41:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:41:27,447 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:41:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:41:37,446 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:41:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:41:47,446 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:41:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:41:57,448 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:42:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:42:07,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:42:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:42:17,446 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:42:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:42:27,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:42:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:42:37,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:42:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:42:47,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:42:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:42:57,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:43:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:43:07,446 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:43:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:43:17,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:43:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:43:27,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:43:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:43:37,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:43:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:43:47,449 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:43:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:43:57,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:44:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:44:07,448 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:44:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:44:17,446 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:44:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:44:27,447 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:44:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:44:37,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:44:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:44:47,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:44:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:44:57,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:45:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:45:07,449 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:45:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:45:17,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:45:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:45:27,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:45:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:45:37,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:45:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:45:47,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:45:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:45:57,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:46:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:46:07,450 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:46:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:46:17,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:46:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:46:27,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:46:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:46:37,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:46:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:46:47,455 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:46:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:46:57,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:47:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:47:07,446 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:47:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:47:17,449 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:47:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:47:27,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:47:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:47:37,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:47:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:47:47,446 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:47:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:47:57,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:48:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:48:07,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:48:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:48:17,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:48:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:48:27,447 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:48:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:48:37,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:48:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:48:47,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:48:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:48:57,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:49:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:49:07,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:49:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:49:17,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:49:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:49:27,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:49:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:49:37,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:49:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:49:47,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:49:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:49:57,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:50:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:50:07,448 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:50:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:50:17,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:50:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:50:27,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:50:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:50:37,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:50:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:50:47,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:50:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:50:57,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:51:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:51:07,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:51:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:51:17,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:51:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:51:27,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:51:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:51:37,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:51:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:51:47,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:51:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:51:57,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:52:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:52:07,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:52:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:52:17,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:52:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:52:27,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:52:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:52:37,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:52:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:52:47,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:52:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:52:57,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:53:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:53:07,449 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:53:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:53:17,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:53:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:53:27,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:53:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:53:37,447 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:53:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:53:47,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:53:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:53:57,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:54:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:54:07,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:54:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:54:17,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:54:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:54:27,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:54:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:54:37,440 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:54:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:54:47,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:54:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:54:57,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:55:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:55:07,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:55:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:55:17,440 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:55:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:55:27,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:55:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:55:37,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:55:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:55:47,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:55:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:55:57,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:56:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:56:07,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:56:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:56:17,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:56:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:56:27,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:56:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:56:37,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:56:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:56:47,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:56:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:56:57,447 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:57:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:57:07,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:57:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:57:17,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:57:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:57:27,440 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:57:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:57:37,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:57:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:57:47,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:57:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:57:57,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:58:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:58:07,447 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:58:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:58:17,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:58:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:58:27,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:58:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:58:37,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:58:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:58:47,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:58:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:58:57,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:59:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:59:07,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:59:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:59:17,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:59:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:59:27,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:59:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:59:37,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:59:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:59:47,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 12:59:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 12:59:57,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:00:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:00:07,449 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:00:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:00:17,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:00:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:00:27,446 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:00:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:00:37,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:00:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:00:47,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:00:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:00:57,446 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:01:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:01:07,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:01:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:01:17,440 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:01:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:01:27,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:01:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:01:37,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:01:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:01:47,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:01:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:01:57,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:02:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:02:07,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:02:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:02:17,446 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:02:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:02:27,453 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:02:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:02:37,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:02:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:02:47,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:02:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:02:57,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:03:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:03:07,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:03:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:03:17,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:03:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:03:27,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:03:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:03:37,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:03:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:03:47,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:03:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:03:57,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:04:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:04:07,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:04:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:04:17,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:04:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:04:27,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:04:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:04:37,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:04:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:04:47,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:04:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:04:57,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:05:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:05:07,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:05:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:05:17,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:05:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:05:27,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:05:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:05:37,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:05:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:05:47,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:05:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:05:57,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:06:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:06:07,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:06:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:06:17,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:06:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:06:27,447 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:06:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:06:37,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:06:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:06:47,449 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:06:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:06:57,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:07:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:07:07,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:07:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:07:17,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:07:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:07:27,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:07:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:07:37,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:07:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:07:47,440 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:07:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:07:57,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:08:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:08:07,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:08:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:08:17,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:08:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:08:27,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:08:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:08:37,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:08:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:08:47,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:08:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:08:57,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:09:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:09:07,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:09:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:09:17,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:09:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:09:27,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:09:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:09:37,440 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:09:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:09:47,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:09:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:09:57,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:10:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:10:07,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:10:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:10:17,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:10:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:10:27,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:10:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:10:37,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:10:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:10:47,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:10:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:10:57,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:11:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:11:07,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:11:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:11:17,440 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:11:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:11:27,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:11:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:11:37,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:11:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:11:47,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:11:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:11:57,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:12:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:12:07,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:12:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:12:17,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:12:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:12:27,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:12:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:12:37,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:12:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:12:47,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:12:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:12:57,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:13:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:13:07,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:13:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:13:17,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:13:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:13:27,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:13:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:13:37,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:13:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:13:47,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:13:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:13:57,446 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:14:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:14:07,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:14:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:14:17,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:14:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:14:27,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:14:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:14:37,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:14:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:14:47,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:14:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:14:57,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:15:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:15:07,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:15:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:15:17,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:15:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:15:27,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:15:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:15:37,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:15:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:15:47,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:15:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:15:57,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:16:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:16:07,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:16:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:16:17,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:16:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:16:27,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:16:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:16:37,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:16:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:16:47,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:16:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:16:57,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:17:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:17:07,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:17:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:17:17,440 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:17:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:17:27,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:17:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:17:37,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:17:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:17:47,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:17:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:17:57,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:18:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:18:07,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:18:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:18:17,446 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:18:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:18:27,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:18:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:18:37,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:18:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:18:47,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:18:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:18:57,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:19:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:19:07,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:19:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:19:17,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:19:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:19:27,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:19:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:19:37,447 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:19:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:19:47,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:19:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:19:57,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:20:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:20:07,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:20:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:20:17,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:20:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:20:27,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:20:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:20:37,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:20:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:20:47,440 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:20:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:20:57,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:21:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:21:07,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:21:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:21:17,440 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:21:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:21:27,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:21:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:21:37,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:21:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:21:47,440 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:21:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:21:57,446 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:22:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:22:07,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:22:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:22:17,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:22:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:22:27,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:22:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:22:37,440 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:22:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:22:47,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:22:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:22:57,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:23:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:23:07,439 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:23:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:23:17,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:23:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:23:27,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:23:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:23:37,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:23:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:23:47,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:23:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:23:57,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:24:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:24:07,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:24:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:24:17,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:24:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:24:27,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:24:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:24:37,446 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:24:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:24:47,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:24:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:24:57,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:25:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:25:07,456 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:25:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:25:17,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:25:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:25:27,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:25:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:25:37,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:25:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:25:47,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:25:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:25:57,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:26:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:26:07,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:26:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:26:17,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:26:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:26:27,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:26:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:26:37,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:26:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:26:47,449 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:26:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:26:57,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:27:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:27:07,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:27:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:27:17,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:27:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:27:27,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:27:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:27:37,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:27:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:27:47,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:27:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:27:57,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:28:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:28:07,447 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:28:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:28:17,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:28:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:28:27,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:28:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:28:37,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:28:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:28:47,455 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:28:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:28:57,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:29:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:29:07,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:29:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:29:17,440 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:29:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:29:27,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:29:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:29:37,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:29:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:29:47,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:29:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:29:57,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:30:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:30:07,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:30:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:30:17,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:30:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:30:27,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:30:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:30:37,440 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:30:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:30:47,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:30:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:30:57,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:31:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:31:07,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:31:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:31:17,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:31:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:31:27,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:31:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:31:37,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:31:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:31:47,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:31:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:31:57,440 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:32:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:32:07,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:32:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:32:17,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:32:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:32:27,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:32:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:32:37,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:32:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:32:47,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:32:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:32:57,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:33:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:33:07,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:33:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:33:17,440 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:33:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:33:27,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:33:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:33:37,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:33:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:33:47,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:33:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:33:57,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:34:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:34:07,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:34:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:34:17,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:34:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:34:27,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:34:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:34:37,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:34:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:34:47,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:34:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:34:57,445 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
vitabaks commented 2 years ago

No problems, the node is secondary from 15:33

I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)

mcankck commented 2 years ago

the problem is actually my machine named PDDCLPSPSQL02SU is in topology and there is no haproxy postgresql service is not running either.

[root@PDDCLPSPSQL02SU ~]# systemctl status patroni.service
● patroni.service - Runners to orchestrate a high-availability PostgreSQL - patroni
   Loaded: loaded (/etc/systemd/system/patroni.service; enabled; vendor preset: disabled)
   Active: active (running) since Mon 2022-02-28 15:33:21 +03; 1h 48min left
 Main PID: 987 (patroni)
    Tasks: 13 (limit: 100623)
   Memory: 203.1M
   CGroup: /system.slice/patroni.service
           ├─  987 /usr/bin/python3.6 /usr/local/bin/patroni /etc/patroni/patroni.yml
           ├─ 1866 /usr/pgsql-14/bin/postgres -D /var/lib/pgsql/14/data --config-file=/var/lib/pgsql/14/data/postgresql.conf --listen_addresses=10.255.117.67,127.0.0.1 --port=5432 --cluster_name=pudo-postgres>
           ├─ 1903 postgres: pudo-postgres-cluster: logger
           ├─ 1911 postgres: pudo-postgres-cluster: startup waiting for 00000002000000000000000A
           ├─ 1919 postgres: pudo-postgres-cluster: checkpointer
           ├─ 1920 postgres: pudo-postgres-cluster: background writer
           ├─ 1921 postgres: pudo-postgres-cluster: stats collector
           ├─ 2057 postgres: pudo-postgres-cluster: postgres postgres [local] idle
           └─22641 postgres: pudo-postgres-cluster: zbx_monitor postgres 127.0.0.1(58014) idle

Feb 28 13:43:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:43:27,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:43:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:43:37,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:43:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:43:47,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:43:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:43:57,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:44:07 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:44:07,444 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:44:17 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:44:17,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:44:27 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:44:27,441 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:44:37 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:44:37,442 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:44:47 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:44:47,443 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)
Feb 28 13:44:57 PDDCLPSPSQL02SU patroni[987]: 2022-02-28 13:44:57,440 INFO: no action. I am (PDDCLPSPSQL02SU), a secondary, and following a leader (PDDCLPSPSQL03SU)

[root@PDDCLPSPSQL02SU ~]# systemctl status postgresql-14.service
● postgresql-14.service - PostgreSQL 14 database server
   Loaded: loaded (/usr/lib/systemd/system/postgresql-14.service; disabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Mon 2022-02-28 12:50:19 +03; 54min ago
     Docs: https://www.postgresql.org/docs/14/static/
 Main PID: 6838 (code=exited, status=1/FAILURE)

Feb 28 12:50:19 PDDCLPSPSQL02SU systemd[1]: Starting PostgreSQL 14 database server...
Feb 28 12:50:19 PDDCLPSPSQL02SU systemd[1]: postgresql-14.service: Main process exited, code=exited, status=1/FAILURE
Feb 28 12:50:19 PDDCLPSPSQL02SU systemd[1]: postgresql-14.service: Failed with result 'exit-code'.
Feb 28 12:50:19 PDDCLPSPSQL02SU systemd[1]: Failed to start PostgreSQL 14 database server.

[root@PDDCLPSPSQL02SU ~]# systemctl restart postgresql-14.service
Job for postgresql-14.service failed because the control process exited with error code.
See "systemctl status postgresql-14.service" and "journalctl -xe" for details.

[root@PDDCLPSPSQL02SU ~]# patronictl topology
+-------------------+---------------+---------+---------+----+-----------+
| Member            | Host          | Role    | State   | TL | Lag in MB |
+ Cluster: pudo-postgres-cluster (7055798345681484839) -+----+-----------+
| PDDCLPSPSQL03SU   | 10.255.117.68 | Leader  | running |  2 |           |
| + PDDCLPSPSQL04SU | 10.255.117.70 | Replica | running |  2 |         0 |
+-------------------+---------------+---------+---------+----+-----------+
[root@PDDCLPSPSQL02SU ~]#
vitabaks commented 2 years ago

The postgresql-14.service service should be disabled and you do not need to run it, because the Patroni service manages of PostgreSQL.

vitabaks commented 2 years ago

It is very strange that the patronictl command does not display the PDDCLPSPSQL02SU node

Check the list of members in the etcd cluster:

etcdctl ls /service/pudo-postgres-cluster/members

mcankck commented 2 years ago 
[root@PDDCLPSPSQL02SU ~]# etcdctl ls /service/pudo-postgres-cluster/members
/service/pudo-postgres-cluster/members/PDDCLPSPSQL03SU
/service/pudo-postgres-cluster/members/PDDCLPSPSQL04SU
vitabaks commented 2 years ago

Run the command on all nodes

@mcankck Are you sure that all nodes work with the same etcd cluster?

mcankck commented 2 years ago

yes, all 3 nodes were in the same etcd cluster. It was installed this way when installing and this is a live environment. No one made any changes to the system. now node 02 seems to be missing

mcankck commented 2 years ago

Is there a way to add that node again? @vitabaks

vitabaks commented 2 years ago

The patroni add node to cluster (etcd)

Your case is not typical and it is not yet clear why this behavior.

I can suggest checking the Patroni configuration and make sure that the scope, name (it must be unique) parameters is correctly specified in the patroni.yml on all nodes.

mcankck commented 2 years ago

It is worth noting that this cluster structure has been standing and working for 3 months, this situation has occurred today without any change or work.

mcankck commented 2 years ago

I have installed this structure many times, with 3 groups on approximately 18 machines, this is the first time I am encountering such a situation.

vitabaks commented 2 years ago

PostgreSQL is working on the problem node?

psql -h localhost -d <dbname>

vitabaks commented 2 years ago

this is the first time I am encountering such a situation.

I haven't encountered it either.

mcankck commented 2 years ago

Yes it works. I can view the databases that have been running so far, but cannot add new ones. because it's not a replica

vitabaks commented 2 years ago

because it's not a replica

Please attach postgres log

mcankck commented 2 years ago

This log 02 named machine ;


2022-02-28 17:10:49 +03 [79474-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:10:54 +03 [79484-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:10:59 +03 [79515-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:11:04 +03 [79528-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:11:09 +03 [79558-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:11:14 +03 [79569-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:11:19 +03 [79598-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:11:24 +03 [79608-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:11:29 +03 [79643-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:11:34 +03 [79653-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:11:39 +03 [79681-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:11:44 +03 [79692-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:11:49 +03 [79720-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:11:54 +03 [79730-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:11:59 +03 [79761-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:12:04 +03 [79774-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:12:09 +03 [79800-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:12:14 +03 [79811-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:12:19 +03 [79840-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:12:24 +03 [79850-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:12:29 +03 [79887-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:12:34 +03 [79897-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:12:39 +03 [79924-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:12:44 +03 [79935-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:12:49 +03 [79964-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:12:54 +03 [80071-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:12:59 +03 [80102-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:13:04 +03 [80115-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:13:09 +03 [80142-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:13:14 +03 [80153-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:13:19 +03 [80181-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:13:24 +03 [80192-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:13:29 +03 [80228-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:13:34 +03 [80238-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:13:39 +03 [80265-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:13:44 +03 [80276-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:13:49 +03 [80304-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:13:54 +03 [80314-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:13:59 +03 [80345-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:14:04 +03 [80358-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:14:09 +03 [80386-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:14:14 +03 [80397-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist

==> /var/log/postgresql/postgresql-Sat.log <==

==> /var/log/postgresql/postgresql-Sun.log <==

==> /var/log/postgresql/postgresql-Thu.log <==

==> /var/log/postgresql/postgresql-Tue.log <==

==> /var/log/postgresql/postgresql-Wed.log <==

==> /var/log/postgresql/postgresql-Mon.log <==
2022-02-28 17:14:19 +03 [80426-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:14:24 +03 [80436-1]  FATAL:  could not start WAL streaming: ERROR:  replication slot "pddclpspsql02su" does not exist

this log master 03 named machine;

2022-02-28 17:15:24 +03 [74555-1] 127.0.0.1(26564) [unknown]@[unknown] LOG:  PID 0 in cancel request did not match any process
2022-02-28 17:15:24 +03 [74563-1] 10.255.117.67(52100) replicator@[unknown] ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:15:24 +03 [74563-2] 10.255.117.67(52100) replicator@[unknown] STATEMENT:  START_REPLICATION SLOT "pddclpspsql02su" 0/A000000 TIMELINE 2
2022-02-28 17:15:29 +03 [74635-1] 10.255.117.67(52142) replicator@[unknown] ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:15:29 +03 [74635-2] 10.255.117.67(52142) replicator@[unknown] STATEMENT:  START_REPLICATION SLOT "pddclpspsql02su" 0/A000000 TIMELINE 2
2022-02-28 17:15:34 +03 [74685-1] 10.255.117.67(52184) replicator@[unknown] ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:15:34 +03 [74685-2] 10.255.117.67(52184) replicator@[unknown] STATEMENT:  START_REPLICATION SLOT "pddclpspsql02su" 0/A000000 TIMELINE 2
2022-02-28 17:15:39 +03 [74754-1] 10.255.117.67(52226) replicator@[unknown] ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:15:39 +03 [74754-2] 10.255.117.67(52226) replicator@[unknown] STATEMENT:  START_REPLICATION SLOT "pddclpspsql02su" 0/A000000 TIMELINE 2
2022-02-28 17:15:44 +03 [74810-1] 10.255.117.67(52264) replicator@[unknown] ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:15:44 +03 [74810-2] 10.255.117.67(52264) replicator@[unknown] STATEMENT:  START_REPLICATION SLOT "pddclpspsql02su" 0/A000000 TIMELINE 2
2022-02-28 17:15:49 +03 [74878-1] 10.255.117.67(52306) replicator@[unknown] ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:15:49 +03 [74878-2] 10.255.117.67(52306) replicator@[unknown] STATEMENT:  START_REPLICATION SLOT "pddclpspsql02su" 0/A000000 TIMELINE 2
2022-02-28 17:15:54 +03 [74932-1] 10.255.117.67(52348) replicator@[unknown] ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:15:54 +03 [74932-2] 10.255.117.67(52348) replicator@[unknown] STATEMENT:  START_REPLICATION SLOT "pddclpspsql02su" 0/A000000 TIMELINE 2

==> /var/log/postgresql/postgresql-Sat.log <==

==> /var/log/postgresql/postgresql-Sun.log <==

==> /var/log/postgresql/postgresql-Thu.log <==

==> /var/log/postgresql/postgresql-Tue.log <==

==> /var/log/postgresql/postgresql-Wed.log <==

==> /var/log/postgresql/postgresql-Mon.log <==
2022-02-28 17:15:59 +03 [75003-1] 10.255.117.67(52390) replicator@[unknown] ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:15:59 +03 [75003-2] 10.255.117.67(52390) replicator@[unknown] STATEMENT:  START_REPLICATION SLOT "pddclpspsql02su" 0/A000000 TIMELINE 2
2022-02-28 17:16:04 +03 [75055-1] 10.255.117.67(52428) replicator@[unknown] ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:16:04 +03 [75055-2] 10.255.117.67(52428) replicator@[unknown] STATEMENT:  START_REPLICATION SLOT "pddclpspsql02su" 0/A000000 TIMELINE 2
2022-02-28 17:16:09 +03 [75127-1] 10.255.117.67(52470) replicator@[unknown] ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:16:09 +03 [75127-2] 10.255.117.67(52470) replicator@[unknown] STATEMENT:  START_REPLICATION SLOT "pddclpspsql02su" 0/A000000 TIMELINE 2
2022-02-28 17:16:14 +03 [75183-1] 10.255.117.67(52512) replicator@[unknown] ERROR:  replication slot "pddclpspsql02su" does not exist
2022-02-28 17:16:14 +03 [75183-2] 10.255.117.67(52512) replicator@[unknown] STATEMENT:  START_REPLICATION SLOT "pddclpspsql02su" 0/A000000 TIMELINE 2
vitabaks commented 2 years ago

The absence of a replication slot is possible if the replica has been unavailable for a long time. When the replica joins the cluster again, the Patroni must create a replication slot itself.

try reinit

patronictl reinit pudo-postgres-cluster PDDCLPSPSQL02SU

mcankck commented 2 years ago

[root@PDDCLPSPSQL03SU ~]# patronictl reinit pudo-postgres-cluster PDDCLPSPSQL02SU +-----------------+---------------+---------+---------+----+-----------+ | Member | Host | Role | State | TL | Lag in MB |

i tried on master machine

vitabaks commented 2 years ago

You can use add_node.yml to retry adding a node to the cluster

But first clear the postgres data directory on the problematic node - PDDCLPSPSQL02SU (do not carefully delete the data on the production server)

sudo systemctl stop patroni
sudo rm -rf /var/lib/postgresql/ # be careful with this if there are other PG clusters

and

ansible-playbook add_pgnode.yml

https://github.com/vitabaks/postgresql_cluster#cluster-scaling

You can also open the issue on the Patroni project repository for resolve this problem.