MattTriano / analytics_data_where_house

An analytics engineering sandbox focusing on real estates prices in Cook County, IL
https://docs.analytics-data-where-house.dev/
GNU Affero General Public License v3.0
9 stars 0 forks source link

Add a maintenance DAG that drops all "temp_" tables from data_raw #64

Closed MattTriano closed 1 year ago

MattTriano commented 1 year ago

The "temp_" tables in data raw are replaced every time the data is updated and aren't used for much if anything outside of the update-DAG context. I've used those tables while developing expectations or _standardized stage models (so I don't necessarily want to automate deletion yet), but before exporting or backing up data, it's a good idea to shed that weight.