Aiven-Open / pghoard

PostgreSQL® backup and restore service
http://aiven-open.github.io/pghoard/
Apache License 2.0
1.32k stars 97 forks source link

monitor pg connection when uploading chunks #634

Open kathia-barahona opened 21 hours ago

kathia-barahona commented 21 hours ago

About this change - What it does

While creating a backup, both pg_start_backup and pg_stop_backup must be executed within the same session. However, in some cases, the backup process may take several hours (especially when processing large chunks). During this time, the PostgreSQL instance may experience a restart, causing the connection used for the backup to be lost.

As a result, pghoard may continue uploading chunks, only to encounter a connection error when attempting to finish the backup. This leads to wasted time, as the system attempts to complete the backup and then raises an exception due to the lost connection. The backup will then retry, potentially leading to further delays and redundant work.

So now, instead of letting the backup continue and fail due to a lost connection, each upload task will actively monitor the connection during the process. If the connection is lost, the task will be aborted, raising an exception to indicate that the connection is no longer available.

Resolves: #BF-2931

Why this way