search

sul-dlss / dlme-airflow

This is a new repository to capture the work related to the DLME ETL Pipeline and establish airflow
Apache License 2.0
1 stars 0 forks source link

DAGs not populating #193

Closed edsu closed 2 years ago

edsu commented 2 years ago

Since 48f5515b1d0f95ac4e1337a3b28547257f04292a Airflow is not instantiating our DAGs. You will just see an empty list on the homepage instead of the list of all the DAGs created from the Catalog.

I think the problem is that Airflow is not discovering the DAGs that are created dynamically in dlme_airflow.dags.harvest_catalog because the DAG classes are now being created in the dlme_airflow.services.harvest_dag_generator module?

edsu commented 2 years ago

Here's a little example to demonstrate how globals() is working https://gist.github.com/edsu/077a4e7089c4ef3ea7de92fd9819ba71