search

ccao-data / data-architecture

Codebase for CCAO data infrastructure construction and management
https://ccao-data.github.io/data-architecture/
5 stars 3 forks source link

Remove deleted `print_s3_dependency_dir` macro call from `cleanup-dbt-resources` workflow #485

Closed jeancochrane closed 3 weeks ago

jeancochrane commented 3 weeks ago

The cleanup-dbt-resources workflow started raising errors after we merged #475 since it was still referencing the print_s3_dependency_dir macro that we deleted as part of that PR. See here for a sample broken workflow run (note that the 404 warning in the logs is irrelevant, since the script is configured to expect those kinds of 404s and continue execution after encountering them). Note that no resources remain in need of being cleaned up from these failed workflow runs, since they all failed on the very last step that has been functionally a no-op since https://github.com/ccao-data/data-architecture/pull/461.

See this workflow run for evidence that this new version of the script executes successfully.

Sidebar: It would be nice if we could figure out a way to unit test this cleanup script, since it has a tendency to break unexpectedly like this, but I can't think of an obvious way to do so. Perhaps we somehow mock the awscli calls and then confirm that the correct list of databases gets expanded into arguments? Curious what you think!

jeancochrane commented 3 weeks ago

Temporarily closing to trigger cleanup-dbt-resources and test the fix.