Datavault-UK / automate-dv

A free to use dbt package for creating and loading Data Vault 2.0 compliant Data Warehouses (powered by dbt, an open source data engineering tool, registered trademark of dbt Labs)
https://www.automate-dv.com
Apache License 2.0
513 stars 131 forks source link

[BUG] pit PKs being generated in mixed case post 0.9.0 #162

Closed versent-tomekwaller closed 2 years ago

versent-tomekwaller commented 2 years ago

[Accidentally clicked new issue too early, I will fill out the rest of the bug report soon, apologies!]

Describe the bug macros/tables/snowflake/pit.sql is providing mixed case PK and LDTS column names when:

his means that when an upper case column already exists, the materialisation is unable to insert into a preexisting table due to an error similar to "SQL compilation error: error line x at position x

invalid identifier 'SAT_SATELLITE_PK".

This seems to be due to looping through "{%- for sat_name in satellites %}", providing a lower case name from the . If an

Environment

dbt version: 1.1 dbtvault version: 0.9.0 Database/Platform: Snowflake / DBT Cloud

To Reproduce Steps to reproduce the behavior:

  1. Already have a dbtvault.pit model/table created using a pit_incremental Materialisation from before 0.9.0. This table will have an upper case "SAT_SATELLITE_PK" and "SAT_SATELLITE_LDTS"
  2. Upgrade to dbt vault 0.9.0
  3. Compile the pit model and the PK now has a mixed column name of "sat_satellite_PK"
  4. When run, the model is unable to find "SAT_SATELLITE_PK" and fails a run.

Expected behavior macros/tables/snowflake/pit.sql should

Screenshots If applicable, add screenshots to help explain your problem.

Log files If applicable, provide dbt log files which include the problem.

Additional context Add any other context about the problem here.

Describe the bug macros/tables/snowflake/pit.sql is providing mixed case PK and LDTS column names when:

his means that when an upper case column already exists, the materialisation is unable to insert into a preexisting table due to an error similar to "SQL compilation error: error line x at position x

invalid identifier 'SAT_SATELLITE_PK".

This seems to be due to looping through "{%- for sat_name in satellites %}", providing a lower case name from the . If an

Environment

dbt version: 1.1 dbtvault version: 0.9.0 Database/Platform: Snowflake / DBT Cloud

To Reproduce Steps to reproduce the behavior:

  1. Already have a dbtvault.pit model/table created using a pit_incremental Materialisation from before 0.9.0. This table will have an upper case "SAT_SATELLITE_PK" and "SAT_SATELLITE_LDTS"
  2. Upgrade to dbt vault 0.9.0
  3. Compile the pit model and the PK now has a mixed column name of "sat_satellite_PK"
  4. When run, the model is unable to find "SAT_SATELLITE_PK" and fails a run.

Expected behavior macros/tables/snowflake/pit.sql should

Screenshots If applicable, add screenshots to help explain your problem.

Log files If applicable, provide dbt log files which include the problem.

Additional context Add any other context about the problem here.

versent-tomekwaller commented 2 years ago

Accidentally clicked new issue too early, I will fill out the rest of the bug report soon, apologies!

versent-tomekwaller commented 2 years ago

Closing to make new cleaner Issue.