Closed jklugherz closed 5 months ago
yeah, just in a brief look at this, my gut is leaning towards option A. The UpdateAnnotations task is already more crowded than it should be (VEP at some point was its own task... but got moved here when we moved things around).
rather than Xcoms I'd recommend a new intermediate file, similar to what we're doing with the WriteImportedCallset
, of just the new variants. It probably also makes sense for VEP to be there!
options for where in pipeline to register the alleles (I implemented option B here):
A) in a new airflow/luigi task downstream of the annotations table tasks
new_variants_ht
)B) within the update annotations table task
new_variants_ht
to get what I need to call the register endpoint with