Aiven-Open / pghoard

PostgreSQL® backup and restore service
http://aiven-open.github.io/pghoard/
Apache License 2.0
1.3k stars 99 forks source link

Fix delta basebackup issue when delta file gets truncated [BF-2146] #590

Closed alexole closed 1 year ago

alexole commented 1 year ago

This fixes an issue when delta backup fails during backup with the following error:

File "/usr/lib/python3.11/site-packages/pghoard/basebackup/delta.py", line 386, in _read_delta_sizes
     snapshot_file.stored_file_size = self.tracked_snapshot_files[snapshot_file.hexdigest].stored_file_size
                                      ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: '69217a3079908094e11121d042354a7c1f55b6482ca1a51e1b250dfd1ed0eef9'

The reason for that is when the uploaded delta file gets truncated during upload, its stored file size is a defalt value of 0, in this case the code assumes that this file was uploaded before and size info needs to be taken from the dict of already uploaded files, which is wrong. Here we just skip override of this attribute for empty delta files.

codecov[bot] commented 1 year ago

Codecov Report

Merging #590 (4115015) into main (85689aa) will increase coverage by 0.20%. The diff coverage is 100.00%.

Additional details and impacted files [![Impacted file tree graph](https://app.codecov.io/gh/aiven/pghoard/pull/590/graphs/tree.svg?width=650&height=150&src=pr&token=nLr7M7hvCx&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aiven)](https://app.codecov.io/gh/aiven/pghoard/pull/590?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aiven) ```diff @@ Coverage Diff @@ ## main #590 +/- ## ========================================== + Coverage 91.11% 91.31% +0.20% ========================================== Files 32 32 Lines 4657 4684 +27 ========================================== + Hits 4243 4277 +34 + Misses 414 407 -7 ``` | [Impacted Files](https://app.codecov.io/gh/aiven/pghoard/pull/590?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aiven) | Coverage Δ | | |---|---|---| | [pghoard/basebackup/delta.py](https://app.codecov.io/gh/aiven/pghoard/pull/590?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aiven#diff-cGdob2FyZC9iYXNlYmFja3VwL2RlbHRhLnB5) | `95.95% <100.00%> (+0.03%)` | :arrow_up: | ... and [11 files with indirect coverage changes](https://app.codecov.io/gh/aiven/pghoard/pull/590/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aiven)