(Archiving) Abort delete file command when file is archived

islean commented 11 months ago

Description

With the new DataFlow, archived files need to be removed using cg archive delete-file since Housekeeper does not communicate with the external API. This PR makes delete-file (delete-files) in Housekeeper abort (skip) files which are archived and log a warning.

Changed

Archived files can no longer be removed via the Housekeeper CLI.

Testing

How to prepare for test

[ ] ssh to Hasta
[ ] Test your branch with

housekeeper-test-deploy archiving-do-not-delete
housekeeper-test <command here>

Any migrations need to be applied manually with alembic against the stage database.

How to test

[ ] login to ...
[ ] do ...

Expected test outcome

[ ] check that ...
[ ] Take a screenshot and attach or copy/paste the output.

Review

[ ] code approved by
[ ] tests executed by
[ ] "Merge and deploy" approved by Thanks for filling in who performed the code review and the test!

This version is a:

[ ] MAJOR - when you make incompatible API changes
[ ] MINOR - when you add functionality in a backwards compatible manner
[ ] PATCH - when you make backwards compatible bug fixes or documentation/instructions

sonarcloud[bot] commented 11 months ago

Quality Gate passed

Kudos, no new issues were introduced!

0 New issues
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

islean commented 11 months ago

Tested in stage:

1d [isak.ohlsson@hasta:/home/proj/stage/housekeeper-bundles] [S_base] 23s $ housekeeper-test delete file 5120426
2023-12-18 15:34:23 hasta.scilifelab.se housekeeper.cli.core[245795] INFO Use root path /home/proj/stage/housekeeper-bundles
2023-12-18 15:34:23 hasta.scilifelab.se housekeeper.cli.delete[245795] WARNING File is archived, please delete it with 'cg archive delete-file' instead
Aborted!
1d [isak.ohlsson@hasta:/home/proj/stage/housekeeper-bundles] [S_base] 15s 1 $

and with delete-files:

1d [isak.ohlsson@hasta:/home/proj/stage/housekeeper-bundles] [S_base] 4s $ housekeeper-test delete files -b isaks_test_bundle
2023-12-18 15:36:44 hasta.scilifelab.se housekeeper.cli.core[247579] INFO Use root path /home/proj/stage/housekeeper-bundles
Are you sure you want to delete 1 files? [y/N]: y
2023-12-18 15:37:43 hasta.scilifelab.se housekeeper.cli.delete[247579] WARNING File isaks_test_bundle/2023-12-18/isaks_test_spring_file.spring is archived, please delete it with 'cg archive delete-file' instead. Skipping.
1d [isak.ohlsson@hasta:/home/proj/stage/housekeeper-bundles] [S_base] 1m3s $

And when the archive entry was deleted:

1d [isak.ohlsson@hasta:/home/proj/stage/housekeeper-bundles] [S_base] 1m3s $ housekeeper-test delete files -b isaks_test_bundle
2023-12-18 15:41:52 hasta.scilifelab.se housekeeper.cli.core[251689] INFO Use root path /home/proj/stage/housekeeper-bundles
Are you sure you want to delete 1 files? [y/N]: y
Remove file from disk and database: /home/proj/stage/housekeeper-bundles/isaks_test_bundle/2023-12-18/isaks_test_spring_file.spring? [y/N]: y
2023-12-18 15:41:56 hasta.scilifelab.se housekeeper.cli.delete[251689] INFO /home/proj/stage/housekeeper-bundles/isaks_test_bundle/2023-12-18/isaks_test_spring_file.spring deleted

islean commented 11 months ago

Deployed to production by IO.

Clinical-Genomics / housekeeper