(Archiving) Archive location via sample

islean commented 3 months ago

Description

We have bugs occurring when customers within a collaboration have different archive locations configured in statusdb. When a case has files archived, the case's customer's archive location is used, but the files are stored on the samples, which might be with a customer with a different archive location. This will all be fixed as soon as the backlog of files are archived, since all clinical customers will share archive location and all collaborators will share the same archive location.

For now, this PR changes the logic to use the samples archive location.

Changed

The sample's customer's archive location is used when retrieving files.

How to prepare for test

[ ] Ssh to relevant server (depending on type of change)
[ ] Use stage: us
[ ] Paxa the environment: paxa

[ ] Install on stage (example for Hasta):

bash /home/proj/production/servers/resources/hasta.scilifelab.se/update-tool-stage.sh -e S_cg -t cg -b retrieve-via-sample -a

How to test

[ ] Do ...

Expected test outcome

[ ] Check that ...
[ ] Take a screenshot and attach or copy/paste the output.

Review

[ ] Tests executed by
[ ] "Merge and deploy" approved by Thanks for filling in who performed the code review and the test!

This version is a

[ ] MAJOR - when you make incompatible API changes
[ ] MINOR - when you add functionality in a backwards compatible manner
[ ] PATCH - when you make backwards compatible bug fixes or documentation/instructions

Implementation Plan

[ ] Document in ...
[ ] Deploy this branch on ...
[ ] Inform to ...

islean commented 3 months ago

Looks a lot cleaner 🧹 Can't seem to find the bug in this code though is it further down the line?

My suspicion is that a case with multiple samples, out of which only a few are archived, might run into some issues. I am pretty certain that the retrieve_sample functionality is expecting a sample with files to archive, so for those samples we will send a request for retrieving zero files. This could raise an error. We should probably raise an error if no archived files are found for the sample which can then be caught in in the case.samples loop.

sonarcloud[bot] commented 3 months ago

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

islean commented 3 months ago

Test successful with the following setup:

Case was created for a customer with PDC archive location.

Sample was created for a customer with karolinska_bucket archive location and linked to the case.

File was created, linked to the sample bundle and marked as archived.

cg archive retrieve case attempted to fetch the file using karolinska_bucket:

[isak.ohlsson@hasta:stage] [S_base] $ cg archive retrieve case IsaksTestCase
/home/proj/stage/bin/miniconda3/envs/S_cg/lib/python3.11/site-packages/urllib3/connectionpool.py:1103: InsecureRequestWarning: Unverified HTTPS request is being made to host 'cg-df-cat.scilifelab.se'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings
  warnings.warn(
Sending request with body: 
{'pathInfo': [{'destination': 'gpfs@Hasta:/proj/stage/housekeeper-bundles/IsaksTestSample/2024-06-17', 'source': 'archive@CGStage:/IsaksTestSample/isaks_test_spring_file.spring'}], 'osType': 'Unix/MacOS', 'createFolder': False, 'settings': [], 'metadataList': []}

islean commented 3 months ago

Deployed to Hasta:

 .bumpversion.cfg           |  2 +-
 cg/__init__.py             |  2 +-
 cg/meta/archive/archive.py | 62 +++++++++++++++++++++++---------------------------------------
 pyproject.toml             |  2 +-
 4 files changed, 26 insertions(+), 42 deletions(-)
/home/proj/stage
INFO  [alembic.runtime.migration] Context impl MySQLImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
[isak.ohlsson@hasta:stage] [P_base] $

Clinical-Genomics / cg