dbt-labs / dbt-codegen

Macros that generate dbt code
https://hub.getdbt.com/dbt-labs/codegen/latest/
Apache License 2.0
484 stars 106 forks source link

Reserved word table names break with quoting off at the project level #43

Closed djbelknapdbs closed 1 year ago

djbelknapdbs commented 3 years ago

Describe the bug

When a table name is a reserved word and you set the generate_columns parameter to True, the generate_source macro fails with an error. You can avoid this by setting quoting at the project level, but that's not recommended especially for Snowflake according to the docs. It's manageable to flip it on during development and back off.

Seems related to #11.

Steps to reproduce

1) Create a table named group 2) Leave quoting unspecified in project.yml 2) Run the generate_source macro on the database/schema that includes the group table and generate_columns=True

Expected results

Produce a source schema entry for the table, or fail more elegantly by skipping the table with a placeholder.

Actual results

The code fails with a Runtime Error - see the log output below.

Screenshots and log output

Encountered an error:
Runtime Error
  Database Error in analysis _codegen (analysis\_codegen.sql)
    001003 (42000): SQL compilation error:
    syntax error line 1 at position 40 unexpected 'GROUP'.
    syntax error line 1 at position 28 unexpected '.'.
    syntax error line 1 at position 39 unexpected '.'.

System information

The contents of your packages.yml file:

  - package: fishtown-analytics/codegen
    version: [">=0.3.0", "<0.4.0"]

Which database are you using dbt with?

The output of dbt --version:

installed version: 0.19.1
   latest version: 0.19.1

Up to date!

Plugins:
  - bigquery: 0.19.1
  - postgres: 0.19.1
  - redshift: 0.19.1
  - snowflake: 0.19.1

The operating system you're using: Windows 10

The output of python --version:

Python 3.8.9

Additional context

I haven't fully traced it but I think it's happening in the call to adapter.get_columns_in_relation. The issue does not happen if you don't set generate_columns=True.

Are you interested in contributing the fix?

Possibly. If I do I'll need a nudge on how to solve the issue. Is there a way to override quoting within a macro?

anthu commented 2 years ago

Hey,

I saw a similar issue reported on Slack and I think it has nothing to do with this package, nor should be fixed here. ~My assumption is that~ it's purely related to the usage of reserved words as identifiers.

Therefore an official solution would be to enable quoting: see recommendation for Snowflake here (not using reserved words - otherwise enable quoting): https://docs.getdbt.com/reference/project-configs/quoting

github-actions[bot] commented 1 year ago

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.

github-actions[bot] commented 1 year ago

Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment to notify the maintainers.