dbt-labs / dbt-project-evaluator

This package contains macros and models to find DAG issues automatically
https://dbt-labs.github.io/dbt-project-evaluator/latest/
Apache License 2.0
447 stars 67 forks source link

Error in `base_source_columns` and `base_node_columns` in Redshift #466

Closed stevenconnorg closed 2 months ago

stevenconnorg commented 4 months ago

Describe the bug

Expected results

Actual results

Screenshots and log output

System information

The contents of your packages.yml file:


packages:
  - package: dbt-labs/audit_helper
    version: 0.12.0
  - package: dbt-labs/dbt_utils
    version: 1.2.0
  - package: dbt-labs/codegen
    version: 0.12.1
  - package: calogica/dbt_expectations
    version: 0.10.3
  - package: elementary-data/elementary
    version: 0.15.2
  - package: dbt-labs/redshift
    version: 0.9.0  
  - package: data-mie/dbt_profiler
    version: 0.8.2
  - package: dbt-labs/dbt_project_evaluator
    version: 0.12.0packages:
  - package: dbt-labs/audit_helper
    version: 0.12.0
  - package: dbt-labs/dbt_utils
    version: 1.2.0
  - package: dbt-labs/codegen
    version: 0.12.1
  - package: calogica/dbt_expectations
    version: 0.10.3
  - package: elementary-data/elementary
    version: 0.15.2
  - package: dbt-labs/redshift
    version: 0.9.0  
  - package: data-mie/dbt_profiler
    version: 0.8.2
  - package: dbt-labs/dbt_project_evaluator
    version: 0.12.0

Which database are you using dbt with?

The output of dbt --version:

(dbt-env-1.8) (dbt-env-1.6) stevengonzalez@Steven-Gonzalez dbt-campaign-workflows % dbt --version
Core:
  - installed: 1.8.3
  - latest:    1.8.3 - Up to date!

Plugins:
  - redshift: 1.8.1 - Up to date!
  - postgres: 1.8.2 - Up to date!

Additional context

Are you interested in contributing the fix?

nevdelap commented 4 months ago

I believe this issue is the same as what I'm seeing in Snowflake. Update: Ah, I see #465. :+1:

dbt-core: 1.8.3 dbt-snowflake: 1.8.3 Python: 3.11

02:51:34    Database Error in model base_source_columns (models/staging/graph/base/base_source_columns.sql)
  002020 (21S01): SQL compilation error:
  Insert value list does not match column list expecting 5 but got 8
  compiled Code at target/run/dbt_project_evaluator/models/staging/graph/base/base_source_columns.sql
b-per commented 4 months ago

Yes, it looks like #465 .

Let's discuss about it there. As we suggested on the other issue, it would be useful to see what SQL is being sent. Because we load data in hooks though it doesn't show under target/... and we can only see them in the dbt logs or in the Data Warehouse query history.

b-per commented 4 months ago

So the issue with base_source_columns is now tracked in #465 and it is getting fixed.

stevenconnorg commented 2 months ago

Thank you! This issue has been resolved.