Clinvar pathogenic filter apparently does not work

[holtgrewe]

Describe the bug A user reported that filtering for "all clinvar pathogenic" variants does not work properly. We get both false positives and false negatives (pathogenic variants are not shown and non-pathogenic variants are shown).

To Reproduce Steps to reproduce the behavior:

1. Filter case with "clinvar pathogenic" filter
2. See non-pathogenic variants and miss pathogenic variants.

Expected behavior The filter results should reflect the clinvar flags that were used when annotating the case.

Screenshots N/A

Additional context This needs careful investigation. Earlier we had a discrepancy between the annotations within the Clinvar database and what was used for annotation. Now with the updated 20210728(b) release the data should be consistent. We can expect inconsistencies with the upstream Clinvar database but that just cannot be helped.

[holtgrewe]

Root Cause Analysis The following confused me a bit in the RCA and led me to look at data with an older release: #497 #498 #499.

Note that we still use the frequency filter. Need to clarify with clinical users about desired behaviour.

I confirmed with the Berlin clinicians that this is not the desired behaviour and we clear the filter. Future version of VarFish will allow local presets.

The query from below can be used to pull some data for looking into this in postgres

SELECT v.chromosome, v.start, v.reference, v.alternative, c.variation_type
FROM variants_smallvariant AS v
LEFT JOIN clinvar_clinvar AS c
ON v.release=c.RELEASE
    AND v.chromosome=c.chromosome
    AND v.start=c.start
    AND v.reference=c.reference
    AND v.alternative=c.alternative
WHERE v.in_clinvar
    AND v.alternative != '*'
    AND v.case_id = 3892
    AND v.set_id = 5634;

A coincidal finding. The following SQL fragment is created: OR NOT flag_for_validation OR where the NOT should not be there.

The point rating etc. is not displayed:

However, I think that the clinvar annotation itself is correct. The main issue appears to be that we are not removing variants classified as benign (yet).

[holtgrewe]

Resolution Proposal

• apparently no real bug fix needed in the queries
• display information in variant fold-out that the upstream clinvar database might have changed
• fix template such that the clinvar significance and rating is displayed properly

Affected Components VarFish Server

Affected Modules/Files Templates in variants module only.

Required Architectural Changes None

Required Database Changes None

Backport Possible? Yes, should be packported to anthenea

Resolution Sketch

• [x] adjust the default frequency filters
• [x] display clinvar significance and rating again
• [x] add informative note regarding possible difference to upstream clinvar

The coincidental finding leads to filtering by flag to be broken in some way, should be fixed in #502, not here.