search

ccao-data / data-architecture

Codebase for CCAO data infrastructure construction and management
https://ccao-data.github.io/data-architecture/
6 stars 4 forks source link

Refactor `res_report_summary` to fix failing daily dbt model builds #653

Closed jeancochrane closed 19 hours ago

jeancochrane commented 4 days ago

Our build-daily-dbt-models workflow is failing every day because reporting.res_report_summary keeps running out of resources:

Query exhausted resources at this scale factor. You may need to manually clean the data at location 's3://ccao-athena-results-us-east-1/tables/ec8046e9-8e2c-4849-81b8-f81a22bc5f6c' before retrying. Athena will not delete data in your account.

Let's refactor the view to reduce the resources that this job uses.