search

EnterpriseDB / barman

Barman - Backup and Recovery Manager for PostgreSQL
https://www.pgbarman.org/
GNU General Public License v3.0
2.06k stars 191 forks source link

Implement a no-list strategy for wal-restore to improve recovery times #877

Open sjuls opened 10 months ago

sjuls commented 10 months ago

Hi @mikewallace1979,

I have another proposal for improving wal-restore download times. This proposal eliminates the list operation in favour of optimistically trying to download the wal file, going through supported compressions and falling back to non-compressed variant.

Since list object operations are quite expensive (eg it's 10x the price of a Get on AWS) and also subject to more variance in performance, this change should be both cheaper and yield a shorter and more consistent download time for all use-cases.

I've run a small experiment to verify whether no-list has the desired effect, and it seems to yield an overall better and more consistent result in my current configuration and cloud provider.

Number of seconds spent to download wal-files - measurements taken against Azure with snappy compression;   List source_dir List wal_path (https://github.com/EnterpriseDB/barman/pull/876) Optimistic no-list
min 6.602070531 4.466363753 4.252058976
max 17.18391819 15.12628784 7.055532052
avg 9.149752375 6.924325793 5.208105985
median 7.51357695 5.796609713 5.204032868
stdev 3.429938044 3.265527009 0.6495604517

I don't have a good way to test the changes against all cloud provider, hope you can assist me here in some fashion.

Let me know what you think.

mikewallace1979 commented 10 months ago

Hi @sjuls - according to the data this looks ilke a worthwhile optimisation!

I wonder if it might also be worth adding an CLI argument, so for cases where the same compression type has been used throughout the archive's lifetime you can specify that compression in the restore_command and Barman will always try that compression first.

I can test this against the remaining two cloud providers though I'm not sure much time I can commit to it this week. If we were to merge this we'll add some corresponding integration tests - unfortunately the integration tests are not public so that's something that'll have to happen behind the scenes for now.

@gcalacoci what do you think about the general idea here?