broadinstitute / seqr-loading-pipelines

hail-based pipelines for annotating variant callsets and exporting them to elasticsearch
MIT License
22 stars 20 forks source link

add allele registry step in update vat with samples task #745

Closed jklugherz closed 5 months ago

jklugherz commented 5 months ago

options for where in pipeline to register the alleles (I implemented option B here):

A) in a new airflow/luigi task downstream of the annotations table tasks

B) within the update annotations table task

bpblanken commented 5 months ago

yeah, just in a brief look at this, my gut is leaning towards option A. The UpdateAnnotations task is already more crowded than it should be (VEP at some point was its own task... but got moved here when we moved things around).

rather than Xcoms I'd recommend a new intermediate file, similar to what we're doing with the WriteImportedCallset, of just the new variants. It probably also makes sense for VEP to be there!