dbt-labs / dbt-project-evaluator

This package contains macros and models to find DAG issues automatically
https://dbt-labs.github.io/dbt-project-evaluator/latest/
Apache License 2.0
432 stars 61 forks source link

macro filter_exceptions calls columns that are not avalailable #381

Closed jschintz-windriver closed 11 months ago

jschintz-windriver commented 11 months ago

Describe the bug

When upgrading from v0.6.1 to v0.7.2 we encountered the following error.

ERROR creating sql table model dbt_cloud_pr_211000_305.fct_direct_join_to_source
000904 (42000): SQL compilation error: error line 58 at position 20
invalid identifier 'PARENT_DIRECTORY_PATH'
compiled Code at target/run/dbt_project_evaluator/models/marts/dag/fct_direct_join_to_source.sql

Steps to reproduce

Line 45 of fct_direct_join_to_source called a macro '{{ filter_exceptions(model.name) }}' This macro compiled to where 1=1 and parent_directory_path = 'XXXX'

But, the field parent_directory_path is not available in upstream 'final' cte

Expected results

all models should execute in package

Actual results

not all models could run in package

System information

The contents of your packages.yml file:

Which database are you using dbt with?

The output of dbt --version: 1.6

Are you interested in contributing the fix?

Sure! Anything I can do to help

b-per commented 11 months ago

Thanks for raising the issue.

Have you overwritten the macro filter_exceptions() in your project? I can't seem to see where where 1=1 and parent_directory_path = 'XXXX' could come from.

jschintz-windriver commented 11 months ago

Thanks for the quick response - no overrides I was using your published package - not even a fork. You've been making such good stuff we want the latest :)

b-per commented 11 months ago

OK, then the issue might be due to the exceptions CSV file you have in place. fct_direct_join_to_source.sql was updated lately to remove some columns and your exception file might still refer to an older columns that was removed.

jschintz-windriver commented 11 months ago

Gotcha! That was it. Really appreciate the help.