Parse and save correctly low-penetrance pathogenic and likely pathogenic ClinVar info

northwestwitch commented 1 week ago

This PR adds a functionality or fixes a bug.

Closes #5048

Testing on cg-vm1 server (Clinical Genomics Stockholm)

**Prepare for testing** 1. Make sure the PR is pushed and available on [Docker Hub](https://hub.docker.com/repository/docker/clinicalgenomics/scout-server-stage) 1. Fist book your testing time using the Pax software available at [https://pax.scilifelab.se/](https://pax.scilifelab.se). The resource you are going to call dibs on is `scout-stage` and the server is `cg-vm1`. 1. `ssh @cg-vm1.scilifelab.se` 1. `sudo -iu hiseq.clinical` 1. `ssh localhost` 1. (optional) Find out which scout branch is currently deployed on cg-vm1: `podman ps` 1. Stop the service with current deployed branch: `systemctl --user stop scout.target` 1. Start the scout service with the branch to test: `systemctl --user start scout@` 1. Make sure the branch is deployed: `systemctl --user status scout.target` 1. After testing is done, repeat procedure at [https://pax.scilifelab.se/](https://pax.scilifelab.se), which will release the allocated resource (`scout-stage`) to be used for testing by other users.

Testing on hasta server (Clinical Genomics Stockholm)

**Prepare for testing** 1. `ssh @hasta.scilifelab.se` 1. Book your testing time using the Pax software. `us; paxa -u -s hasta -r scout-stage`. You can also use the WSGI Pax app available at [https://pax.scilifelab.se/](https://pax.scilifelab.se). 1. (optional) Find out which scout branch is currently deployed on cg-vm1: `conda activate S_scout; pip freeze | grep scout-browser` 1. Deploy the branch to test: `bash /home/proj/production/servers/resources/hasta.scilifelab.se/update-tool-stage.sh -e S_scout -t scout -b ` 1. Make sure the branch is deployed: `us; scout --version` 1. After testing is done, repeat the `paxa` procedure, which will release the allocated resource (`scout-stage`) to be used for testing by other users.

How to test:

how to test it, possibly with real cases/data

Expected outcome: The functionality should be working Take a screenshot and attach or copy/paste the output.

Review:

[x] code approved by DNIL
[x] tests executed by CR

codecov[bot] commented 1 week ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 84.78%. Comparing base (8a9d03a) to head (9e6516e). Report is 1 commits behind head on main.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #5062 +/- ## ======================================= Coverage 84.78% 84.78% ======================================= Files 323 323 Lines 19432 19442 +10 ======================================= + Hits 16475 16484 +9 - Misses 2957 2958 +1 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

🚨 Try these New Features:

Flaky Tests Detection - Detect and resolve failed and flaky tests
JS Bundle Analysis - Avoid shipping oversized bundles

northwestwitch commented 1 week ago

Tested by re-loading the case that triggered the warning

Loading without warnings:

Info saved in the database alright:

I just have to figure out why the info is not displayed on the variant page now.. 🤔

northwestwitch commented 1 week ago

Ah I'm an idiot, I wasn't thinking that in order to see a change in the web page I have to deploy it also on cg-vm1. Here it is:

This is the variant in question ---> https://scout-stage.scilifelab.se/cust000/SWEDAC-2019-HG-05-Trio-NovaSeq/39e574903c122fcd127000368be42330

dnil commented 4 days ago

Looks good to me!

What does this mean for previous cases? We have represented all "low_penetrance" as "other"? They would still have been loaded, due to the keyword "pathogenic" in the string, right? And is this new from this year, or have we had it longer?

Ah, no, its rather that the "low penetrance" was just not represented at all, only with the main term, ie "Pathogenic" for a "Pathogenic, low penetrance"?

northwestwitch commented 4 days ago

Ah, no, its rather that the "low penetrance" was just not represented at all, only with the main term, ie "Pathogenic" for a "Pathogenic, low penetrance"?

Precisely. When you had "Pathogenic, low penetrance", then the parsing of the second part after the comma failed, generating the warning when the variants were loaded. Only "Pathogenic" was saved under clnsig of the variant, which is correct but also incomplete info I think.

If you check the ClinVar pages, pathogenic LP and likely pathogenic LP are categories apart:

So it would make sense to assign a specific category to these 2 as well. The biggest problem is that there is no corresponding int to these categories, like you have for pathogenic (5) or likely pathogenic (4). Of course you could create one but it would be arbitrary and disconnected from ClinVar, where the ints were originally taken from.

I feel this way is safe because we are not altering what is saved into the database for the variant (4 and 5), but we just add more info.

northwestwitch commented 4 days ago

I could make this PR a patch perhaps, so we deploy it with the rest?