opentargets / orchestration

Open Targets pipeline orchestration layer
Apache License 2.0
1 stars 0 forks source link

feat: add interface with batch operators and refactor the utils #7

Closed ireneisdoomed closed 4 months ago

ireneisdoomed commented 4 months ago

✨ Context

Addition of Google Batch functions using #678 as reference and refactor of the airflow utils we had in common_airflow to interact with the operators into 3 separate modules:

For reference on how to use the Batch interactors, use the variant index DAG https://github.com/opentargets/gentropy/blob/dfae8582cce328b5f91ff9edb3cb360814318943/src/airflow/dags/variant_index.py

πŸ›  What does this PR implement

πŸ™ˆ Missing

Test run of a DAG

🚦 Before submitting