dbt-labs / dbt-project-evaluator

This package contains macros and models to find DAG issues automatically
https://dbt-labs.github.io/dbt-project-evaluator/latest/
Apache License 2.0
453 stars 69 forks source link

Error in `base_source_columns` #465

Closed nam-97 closed 4 months ago

nam-97 commented 5 months ago

Describe the bug

When running the model base_source_columns by dbt run, I encounter below error image

Steps to reproduce

Expected results

Actual results

Screenshots and log output

System information

The contents of your packages.yml file:

Which database are you using dbt with?

The output of dbt --version:

1.8.1

Additional context

The problem locates on the post-hook parameter in the model, since the query to create table still can be run in Snowflake. Probably, the post-hook query is trying to insert more columns than the ones existing in the output object image

Are you interested in contributing the fix?

b-per commented 5 months ago

Thanks for raising this.

Would you be able to check the dbt log file and/or Snowflake logs to find the exact statement that failed?

kverburg commented 4 months ago

Also got this error in BQ. Error is there: Inserted row has wrong column count; Has 8, expected 5 at [3:1]

Hereby the exact query the db got. Did trim it a bit, but the gest is still there.

/* {"app": "dbt", "dbt_version": "1.8.1", "profile_name": "dbt_project", "target_name": "test", "node_id": "model.dbt_project_evaluator.base_source_columns"} */
INSERT INTO
    `project`.`dataset`.`base_source_columns`
VALUES
    (
        '''source.dbt_project.dbt_source.source_table''',
        '''_airbyte_raw_id''',
        '''''',
        '''string''',
        '''[]''',
        FALSE,
        0,
        '''None'''
    ),
    ...
    (
        '''source.dbt_project.dbt_source.source_table''',
        '''_airbyte_extracted_at''',
        '''''',
        '''timestamp''',
        '''[]''',
        FALSE,
        0,
        '''None'''
    );
b-per commented 4 months ago

Thanks a lot!

OK, so I can definitely see the error now. This is a bug in the code but we are actually not testing it in CI. I will add a new test and release a new patch version soon.

b-per commented 4 months ago

0.12.1 should be available in a bit in the dbt Hub