dbt-labs / dbt-project-evaluator

This package contains macros and models to find DAG issues automatically
https://dbt-labs.github.io/dbt-project-evaluator/latest/
Apache License 2.0
451 stars 68 forks source link

Bug in Overriding primary_key_test_macros Variable #470

Closed savannahchunn closed 4 months ago

savannahchunn commented 4 months ago

Describe the bug

Overriding the primary_key_test_macros variable to a set of tests that does not include a reference to the dbt.test_unique causes an unresolved column error in the int_model_test_summary model.

Steps to reproduce

At first, I set the primary_key_test_macros variable to be just the below boxed section of our team's custom unique test and the built-in dbt not null test. image

Expected results

I expected these two tests to be an acceptable set for the primary key check.

Actual results

The use of just the boxed test macros resulted in the below column error where int_mdoel_test_summary was looking for a is_test_unique column in the all_graph_resources model, but due to my team's custom unique test, only a is_test_unique_full column was created in all_graph_resources. image

Screenshots and log output

Screenshots seen above.

System information

The contents of your packages.yml file: packages:

Which database are you using dbt with?

The output of dbt --version: image

Additional context

I think the below line 18 of int_model_test_summary is causing the issue, but I'm not sure if this is_test_unqiue setup is needed by other project evaluator processes. image

Instead of trying to change the project evaluator code, I took a band-aid approach by updating our primary_key_test_macros variable to include a second set of tests, the dbt_utils.test_unqiue_combination_of_columns and the dbt.test_unqiue macros. Having the dbt unqiue test resolves the unknown column error I was getting, but it does open up the slim chance that models in our project pass the evaluator check with a testing approach different than our team's standard. image

Are you interested in contributing the fix?

Not at this time, but I'm happy to provide more context if needed.

b-per commented 4 months ago

Thanks for the report. I will have a look at it today.