edanalytics / edu_edfi_airflow

Manages extract-load of Ed-Fi data in Airflow
Other
4 stars 0 forks source link

Bugfix/check that match rates were calculated before reloading #67

Closed sleblanc23 closed 3 weeks ago

sleblanc23 commented 3 weeks ago

Fixes an issue with the match_rates_to_snowflake task, which would run even if new match rates had not been calculated. This would delete existing match rates with nothing to replace them, requiring them to be recalculated on the next run.

The task now checks that a match rates file exists before doing the delete + load. Tested and working well in Denver.