ClusterLabs / PAF

PostgreSQL Automatic Failover: High-Availibility for Postgres, based on Pacemaker and Corosync.
http://clusterlabs.github.io/PAF/
Other
340 stars 55 forks source link

PAF, Pacemaker, Postgres 15 and replication slots #223

Closed andrecowie closed 1 year ago

andrecowie commented 1 year ago

I am trying to understand how replication slots work when using Pacemaker and PAF.

Obviously I am using a new postgres version that I assume PAF and Pacemaker are not ready for.

I wish to understand what will happen with replication in the case of a failover. From my primitive understanding, PAF doesn't manage primary_slot_names for postgres.

So let's say we have a three node cluster.

node1 (active) node2 (standby) node3 (standby)

Lets say I create two replication slots for node2 and node3 select pg_create_physical_replication_slot('node2'); select pg_create_physical_replication_slot('node3');

And have the standby.signal file in the data directory of node2 and node3.

So I start the all three nodes and I have a successfully configured an active standby standby cluster.

But then comes the part I cannot wrap my head around.

There is no replication_slot for node1 in case of failover. (assuming pacemaker will handle failover on postgres 15) If I create a replication_slot for node1 then the wal will fill up the disk waiting for node1 to come online if it ever does failover (my understanding).

If lets say for instance the node1 the active node fails for whatever reason I assume that pacemaker and PAF will be able to make one of the standby nodes the active node in postgres 15.

However I also assume that node1 will just stay in a strange state where it doesnt get a replication slot assigned and just stays useless until manual intervention.

I guess this issue is just a call to improve documentation into how PAF plays with newer versions of postgres and how it handles physical_replication_slots. As it says on the readme: 'PAF supports PostgreSQL 9.3 and higher' so I would love to get a discussion going on this topic. I am knew to pacemaker and corosync so if I am missing something that can be pointed out that would be great.

I will just assume that I should just run an active standby standby postgres cluster and not configure pacemaker.

I will also try to get time to test failover scenarios in this postgres15 cluster with pacemaker setup to see how it behaves.

ioguix commented 1 year ago

Hi @andrecowie ,

PAF doesn't pay attention to the slots, at all, by design. PAF is responsible to keep your service up and running as long as possible. Slots are risky for availability as if a standby fails for a long time, WALs piles up in pg_wal indefinitely, until filling up your disk.

So creating and managing slot imply to monitor them and potentially drop them if the standby fails for a long time. This makes the resource agent code more complex.

Theoretically, we could create/drop them during promote/demote action and forget about them since max_slot_wal_keep_size has been added in v13... But then you have to check for this parameter value.

But the real reason we don't want to deal with slots is that there's a better and expected architecture: WAL archiving. Step one for a reliable and high available cluster are PITR backups. As long as you have PITR backups, you have WAL archives. As long as you have WAL archives reachable from your standby, you just don't need slots.

Regards,