search

stackabletech / airflow-operator

Stackable Operator for Apache Airflow
Other
21 stars 2 forks source link

Investigate possible race conditions when gitsync-ing DAG submodules #417

Open adwk67 opened 5 months ago

adwk67 commented 5 months ago

Description

As a user of Airflow I want to be sure that DAGs that use submodules are seamlessly updated and registered with Airflow via gitsync. This covers the following:

### Tasks
- [ ] Check that DAGs are consistent i.e. that the state when the DAG is started is the same despite subsequent syncs while the DAG is still running
- [ ] Check that "new" submodules are correctly loaded/registered