dbt-labs / dbt-project-evaluator

This package contains macros and models to find DAG issues automatically
https://dbt-labs.github.io/dbt-project-evaluator/latest/
Apache License 2.0
416 stars 59 forks source link

Initial build fails with column size errors #463

Open mihirshah-sg opened 1 month ago

mihirshah-sg commented 1 month ago

Describe the bug

When running dbt build --select package:dbt_project_evaluator, I get a few errors when the package tries to run staging models related to column size.

Steps to reproduce

This is currently what is in my dbt_project.yml:

  dbt_project_evaluator:
    +enabled: "{{ env_var('ENABLE_DBT_PROJECT_EVALUATOR', 'true') | lower == 'true' | as_bool }}"
    +tags: "Analytics Engineering"
    +meta:
      +owner: "Analytics Engineering"
    marts:
      core:
        enabled: true
      dag:
        enabled: false
        fct_direct_join_to_source:
          +enabled: false
        fct_model_fanout:
          +enabled: false
        fct_multiple_sources_joined:
          +enabled: false
        fct_source_fanout:
          +enabled: false
        fct_staging_dependent_on_staging:
          +enabled: false
      documentation:
        fct_documentation_coverage:
          +enabled: true
        fct_undocumented_models:
          +enabled: true
      performance:
        enabled: false
      structure:
        enabled: false
        fct_model_directories:
          +enabled: false
        fct_model_naming_conventions:
          +enabled: false
      tests:
        enabled: false
        fct_missing_primary_key_tests:
          +enabled: false
        fct_test_coverage:
          +enabled: false
      dispatch:
        - macro_namespace: dbt
          search_order: ['dbt_project_evaluator', 'dbt']

I also have a few lines disabling the package's seed file (to use my own) and enabling tests.

Expected results

All models run.

Actual results

Models do not run.

Screenshots and log output

Screenshot 2024-06-11 at 12 53 05 PM

System information

The contents of your packages.yml file:

  - package: dbt-labs/dbt_project_evaluator
    version: 0.6.0

Which database are you using dbt with?

The output of dbt --version:

Core:
  - installed: 1.3.1
  - latest:    1.8.2 - Update available!

  Your version of dbt-core is out of date!
  You can find instructions for upgrading here:
  https://docs.getdbt.com/docs/installation

Plugins:
  - postgres: 1.3.7 - Update available!
  - redshift: 1.3.1 - Update available!

  At least one plugin is out of date or incompatible with dbt-core.
  You can find instructions for upgrading here:
  https://docs.getdbt.com/docs/installation

Are you interested in contributing the fix?

No

b-per commented 1 month ago

Thanks for the report.

I can see that you are on a pretty old version of the package though and we did quite a few changes recently on how we handle string/varchar in Redshfit (for example like here)

Would you be able to upgrade the package version? This will require also upgrading your dbt version most likely so I don't know if it is possible for you.

stevenconnorg commented 4 weeks ago

I'm also running into this error, and made a work-around by changing the dbt.type_string() calls to dbt.type_large_string() in models base_node_columns and base_source_columns

edit: I'm running dbt v1.8 in Redshift with dbt project evaluator v0.12.0