buda-base / ao-workflows

Use DAG platform to define and orchestrate workflows
0 stars 0 forks source link

Remove syncd works from airflow #24

Closed jimk-bdrc closed 2 months ago

jimk-bdrc commented 2 months ago

After a work has been sync'd, its debagged contents persist in airflow://home/airflow/bdrc/data which is mapped on a real fs to AIRFLOW_DIR/AO-staging/Incoming Rather than manually remove them, a successful sync should remove the source dir. The sync debagged task has access to this.