populationgenomics / automated-interpretation-pipeline

Rare Disease variant prioritisation MVP

MIT License

5 stars 4 forks source link

Updates to ClinVar parsing #311

Closed MattWellie closed 9 months ago

MattWellie commented 9 months ago

Fixes

See Slack Discussion here
Site filtering included a typo, so we were not correctly filtering out our own ClinVar submissions
Report Hunter is still vulnerable to any web analysis entries which aren't HTML files
Even new ClinVar entries can be un-dated, which will affect all date filtration we do

Proposed Changes

If we run any date filtering, remove all un-dated entries from ClinVar (instead of allowing all undated entries to pass all date thresholds)
Correct the site blacklisting, and publish to logs to clarify the exact sites being removed
Correct Web Analysis entry parsing, allowing for soft failures