search

EnterpriseDB / barman

Barman - Backup and Recovery Manager for PostgreSQL
https://www.pgbarman.org/
GNU General Public License v3.0
2.06k stars 191 forks source link

Recovery does not include partial wal file unless --get-wal is specified which implies that SSH is enabled #393

Open allanharry opened 3 years ago

allanharry commented 3 years ago

Hi Due to security we dont want that database team can gain acces to backups in BARMAN using SSH. So we are using streaming protocol for wal files and pgbasebackup where ssh is not needed.

For the recover part we observe that transactions in partial wal file not being applied ! barman recover \ --remote-ssh-command "sshpass -p xxxxxx ssh postgres@pg1" \ pg1 \ 20210124T232937 \ /pg_database/pg12

Doing the recover with --get-wal and allowing ssh from pg1 to barman server will include partial wal file and RPO=zero: barman recover \ --get-wal \ --remote-ssh-command "sshpass -p xxxxxx ssh postgres@pg1" \ pg1 \ 20210124T232937 \ /pg_database/pg12

If backup can be done using streaming protocol then this partial wal file should also somehow be handled during the recover to ensure RPO=zero. Are there any plans to include this in BARMAN ?

Regards Allan

nlgtuankiet commented 2 years ago

I can confirm that the issue exists. @mikewallace1979 is there any update on this?

mikewallace1979 commented 2 years ago

@nlgtuankiet Thanks for confirming - I have also reproduced this. We'll get a fix into one of the upcoming releases.

pct-cclausen commented 2 years ago

Also ran into this, not realizing I will need a ssh connection from postgres to barman to restore partial wal files . Since my database is sitting inside a docker container that doesn't have an ssh client this complicates the setup to restore data significantly for me :(

rprovodenko commented 3 months ago

This issue has been open for 3 years now. Any workaround at all please?

martinmarques commented 2 months ago

The workaround is to manually copy the partial WAL into the barman_wal directory. I am triaging this ticket, but the ask from the OP was to provide streaming from the barman server to the postgres server. That seems like something to implement in postgres itself.

martinmarques commented 2 months ago

IMO the solution would be for barman to recognize that there is a partial WAL (maybe we should also have a flag in barman recover asking for the partial file, in case someone would prefer it not to be used) and copy it together with the WAL files during the barman recover execution.