dbt-labs / dbt-project-evaluator

This package contains macros and models to find DAG issues automatically
https://dbt-labs.github.io/dbt-project-evaluator/latest/
Apache License 2.0
440 stars 64 forks source link

databricks: print_dbt_project_evaluator_issues - quoting #399

Closed bram-83 closed 10 months ago

bram-83 commented 10 months ago

Describe the bug

Printing the results does not work when the databricks schema contains special characters

Steps to reproduce

using special characters in your schema: for example ch-123

Expected results

the expected behaviour = printing the results

Actual results

on-run-end failed, error:

[INVALID_IDENTIFIER] The identifier ch-51278 is invalid. Please, consider quoting it with back-quotes as ch-51278.(line 4, pos 28) == SQL == / {"app": "dbt", "dbt_version": "1.7.3", "dbt_databricks_version": "1.7.2", "databricks_sql_connector_version": "2.9.3", "profile_name": "user", "target_name": "qas", "connection_name": "master"} /

    select * from qas.ch-51278.fct_test_coverage

----------------------------^^^

Screenshots and log output

System information

The contents of your packages.yml file: packages:

Which database are you using dbt with?

The output of dbt --version: Running with dbt=1.7.3 Registered adapter: databricks=1.7.2

Additional context

dbt_project_evaluator > macros > on-run-end > print_dbt_project_evaluator_issues set db_schema: quote a lowercase the three level namespace

b-per commented 10 months ago

Hi @bram-83 , thanks for raising this.

As we don't use ref in the current code, it doesn't take in account the quoting config that might be set in dbt_project.yml.

What I would recommend is doing a copy of the macro and changing this line from

{% set model_schema = model_details.schema %}

to

{% set model_schema = "`" ~ model_details.schema ~ "`"  %}

You can then update you on-run-end to point to your own macro instead of the package one.

If this works, we could update the macro in the package and add an optional parameter to add quoting.

Let me know how it goes.

bram-83 commented 10 months ago

Hi @b-per, Thank you for the tip. I quoted the schema & catalog name. I't working fine now, thanks, image

b-per commented 10 months ago

When we do a new release you will be able to use print_dbt_project_evaluator_issues(quote='`')