gwpy / gwsumm

Gravitational-wave interferometer summary information system
GNU General Public License v3.0
12 stars 23 forks source link

Safer archive process #378

Closed iaraota closed 12 months ago

iaraota commented 1 year ago

It changes the structure of the archiving function. Instead of moving the current backup to a temporary file, it now creates a temporary file for the new saved data. This new file is only moved to its proper place if there are no issues, making sure the backup is secure, even if there's a saving error.

This PR replaces #372.

codecov[bot] commented 1 year ago

Codecov Report

Merging #378 (4d5884e) into master (958795b) will decrease coverage by 0.02%. The diff coverage is 77.78%.

@@            Coverage Diff             @@
##           master     #378      +/-   ##
==========================================
- Coverage   50.27%   50.25%   -0.02%     
==========================================
  Files          60       60              
  Lines        8677     8670       -7     
==========================================
- Hits         4362     4357       -5     
+ Misses       4315     4313       -2     
Flag Coverage Δ
Linux 50.25% <77.78%> (-0.02%) :arrow_down:
python3.10 50.25% <77.78%> (-0.02%) :arrow_down:
python3.7 49.45% <77.78%> (-0.02%) :arrow_down:
python3.8 50.25% <77.78%> (-0.02%) :arrow_down:
python3.9 50.25% <77.78%> (-0.02%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
gwsumm/archive.py 71.81% <77.78%> (+0.01%) :arrow_up:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

eagoetz commented 12 months ago

Per @iaraota, the procedure used to test this is as follows:

  1. Activate the environment that runs gwsumm with the changes
  2. Run a gwsumm process (I basically reran a single day of PCAL, where I copied over the archive and output to my home directory)
  3. Check if the file was created as expected: h5ls -r day/20230918/archive/L1-PCAL-1379030418-86400.h5
  4. Run the same gwsumm process to check if it loads the data as expected
  5. Corrupt the data with by adding the following (before the line h5file.visititems(lambda name, obj: None))
    h5file.write(b'\x00\x01\x02\x03\x04')
  6. Run the same gwsumm process to check that it correctly print the warning message
  7. Check if the previous archive is still there: h5ls -r day/20230918/archive/L1-PCAL-1379030418-86400.h5
  8. Remove the corruption snipet
  9. Run the same process to check if it loads the data as expected

I confirm it works as advertized