NDC Description mart is currently only manually triggered. It depends on 5 different Airflow DAGs.
Same for our ATC to RxNorm mart.
Criteria for Success
Marts are appropriately triggered weekly (NOTE: mart could probably be triggered as frequently as daily b/c the FDA sources update daily - I am currently only delivering an updated mart flatfile weekly, which is why I initially said weekly here).
Dependent data sources are refreshed in a timely manner to ensure mart has the most up-to-date data possible.
Errors or failed tests are handled gracefully and we are aware of them. This might end up being a separate issue - we had a Slack notification system, but it has since fallen into disrepair.
Additional Information
We've considered two approaches to this. There may be other, better ones.
Create an Airflow DAG called marts that just schedules dbt build actions.
Trigger mart build at the end of each of the 5 (or whatever) dependent DAGs.
Problem Statement
NDC Description mart is currently only manually triggered. It depends on 5 different Airflow DAGs.
Same for our ATC to RxNorm mart.
Criteria for Success
Additional Information
We've considered two approaches to this. There may be other, better ones.
martsthat just schedules dbt build actions.