search

ccao-data / data-architecture

Codebase for CCAO data infrastructure construction and management
https://ccao-data.github.io/data-architecture/
5 stars 3 forks source link

Fix broken categorization in `transform_dbt_test_results.py` and remove `dbt_utils` dependency #474

Closed jeancochrane closed 1 month ago

jeancochrane commented 1 month ago

I noticed today that the categorization of tests performed by transform_dbt_test_results.py broke after we changed the name of our dbt project from athena to ccao_data_athena in https://github.com/ccao-data/data-architecture/pull/422, causing most of our tests to get incorrectly categorized as miscellaneous. This PR fixes that bug and removes the coupling of transform_dbt_test_results.py to the dbt project name so that we can change the name of the project without breaking the script in the future.

I also searched the repo to double-check that there aren't any more hidden references to the old project name hanging around, and as far as I can tell this is the last one.

Note that this PR also removes the dependency on dbt_utils, since I noticed that the last remaining use of a dbt_utils macro could be refactored to use one of our macros. We've been following a de facto pattern of vendoring dbt_utils macros since we need them to support our additional_select_columns interface, and this change makes that pattern official.

I tested this by rerunning transform_dbt_test_results.py and confirming that the test categorization works as expected again. I'm happy to share the output if you would like to double-check.