Clinical-Genomics / scout

VCF visualization interface
https://clinical-genomics.github.io/scout
BSD 3-Clause "New" or "Revised" License
152 stars 46 forks source link

Fix filters parse and display #5061

Closed dnil closed 3 days ago

dnil commented 1 week ago

This PR adds a functionality or fixes a bug.

Bit of a difference already without touching parsing: Screenshot 2024-11-22 at 10 44 30

Then with parsing fixed: Screenshot 2024-11-22 at 14 21 35

Screenshot 2024-11-22 at 14 34 12

I also added a filters badge to the variant page for good measure: Screenshot 2024-11-22 at 14 49 06

Testing on cg-vm1 server (Clinical Genomics Stockholm) **Prepare for testing** 1. Make sure the PR is pushed and available on [Docker Hub](https://hub.docker.com/repository/docker/clinicalgenomics/scout-server-stage) 1. Fist book your testing time using the Pax software available at [https://pax.scilifelab.se/](https://pax.scilifelab.se). The resource you are going to call dibs on is `scout-stage` and the server is `cg-vm1`. 1. `ssh @cg-vm1.scilifelab.se` 1. `sudo -iu hiseq.clinical` 1. `ssh localhost` 1. (optional) Find out which scout branch is currently deployed on cg-vm1: `podman ps` 1. Stop the service with current deployed branch: `systemctl --user stop scout.target` 1. Start the scout service with the branch to test: `systemctl --user start scout@` 1. Make sure the branch is deployed: `systemctl --user status scout.target` 1. After testing is done, repeat procedure at [https://pax.scilifelab.se/](https://pax.scilifelab.se), which will release the allocated resource (`scout-stage`) to be used for testing by other users.
Testing on hasta server (Clinical Genomics Stockholm) **Prepare for testing** 1. `ssh @hasta.scilifelab.se` 1. Book your testing time using the Pax software. `us; paxa -u -s hasta -r scout-stage`. You can also use the WSGI Pax app available at [https://pax.scilifelab.se/](https://pax.scilifelab.se). 1. (optional) Find out which scout branch is currently deployed on cg-vm1: `conda activate S_scout; pip freeze | grep scout-browser` 1. Deploy the branch to test: `bash /home/proj/production/servers/resources/hasta.scilifelab.se/update-tool-stage.sh -e S_scout -t scout -b ` 1. Make sure the branch is deployed: `us; scout --version` 1. After testing is done, repeat the `paxa` procedure, which will release the allocated resource (`scout-stage`) to be used for testing by other users.

How to test:

  1. check a cancer case and an RD case, note filter tags missing and tags saying Pass when they should say Filtered.
  2. apply patch
  3. note improvements with old cases, e.g. some new danger badges on variantS, and the filters badge on the variant page. And in particular no new issues with them.
  4. for full effect, load/update the cases with the patch and note good tags both for callers and filters

Expected outcome: The functionality should be working Take a screenshot and attach or copy/paste the output.

Review:

dnil commented 1 week ago

Ok, with that out of the world, I would say we are not formally bugged at least not based on the demo case situation. Screenshot 2024-11-22 at 10 53 13 Since the set tag is used without a "filteredIn", it is hard to fault the parsing here. This looks like more like a merging/pipeline issue.

But we could perhaps do slightly better here: since there is only one caller given, we could assign the FILTER status to the call I think!

codecov[bot] commented 1 week ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 84.81%. Comparing base (53f537b) to head (95ee422). Report is 1 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #5061 +/- ## ========================================== + Coverage 84.78% 84.81% +0.02% ========================================== Files 323 323 Lines 19442 19475 +33 ========================================== + Hits 16484 16517 +33 Misses 2958 2958 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

northwestwitch commented 4 days ago

Just to understand, when there is filter passed but not a specific caller, then originally the filter was not "Passed" but "filtered"?

dnil commented 4 days ago

I'm still trying to understand why we still have the green PASS label with or without callers for some cancer variants. I guess I have to look inside the VCF

Yes, there are some lines there that appear odd to my understanding, much like if it was an amalgamation of different pipeline runs or call sets. I guess the demo is a) old and b) perhaps not entirely representative of one pipeline version, but has some edited lines etc.

dnil commented 4 days ago

Just to understand, when there is filter passed but not a specific caller, then originally the filter was not "Passed" but "filtered"?

Hm, no, I thought it would have been Pass and None on all callers for the category?

I think you are clear on this, but to be sure, and to remember later, the issues were in particular

  1. If a filter status was new/missing from a scout constant VARIANT_FILTERS of known (mostly Lund specific) filter strings, the filter status badge was not shown.
  2. For variants with variant.FILTER status, callers are sometimes marked, e.g. with INFO.FOUND_IN. That was likely not what we expected from the pipeline side, but I guess it makes some sense as well, to tell which tool made the call that got filtered. These would previously be marked with caller status Pass even if a variant.FILTER status was set.
sonarcloud[bot] commented 3 days ago

Quality Gate Passed Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud