search

Datavault-UK / automate-dv

A free to use dbt package for creating and loading Data Vault 2.0 compliant Data Warehouses (powered by dbt, an open source data engineering tool, registered trademark of dbt Labs)
https://www.automate-dv.com
Apache License 2.0
478 stars 114 forks source link

[BUG] Ghost record generation produces invalid code when satellite hashdiff has an alias #173

Closed mjahammel closed 1 year ago

mjahammel commented 1 year ago

Describe the bug When the hashdiff column for a satellite is specified with an alias, the ghost record generation fails with an error like so:

16:48:36  Database Error in model sat_ovtn_member_pii (models/raw_vault/satellites/sat_ovtn_member_pii.sql)
16:48:36    000904 (42000): SQL compilation error: error line 62 at position 33
16:48:36    invalid identifier 'G.DV_HASHDIFF_SAT_OVTN_MEMBER_PII'

Examining the generated code shows that the 'ghost' CTE has not included the hashdiff column, resulting in the error.

Versions dbt: 1.3.0 dbtvault: 0.9.1 Target: Snowflake

To Reproduce Steps to reproduce the behavior:

  1. Set up hashdiff column with alias in the satellite definition: 
    src_hashdiff:
source_column: DV_HASHDIFF_SAT_OVTN_MEMBER_PII
alias: DV_HASHDIFF
  2. Add the variables to generate the ghost record in dbt_project.yml: 
    enable_ghost_records: true
system_record_value: "GHOST"
  3. Run dbt to generate the satellite
  4. See error

Expected behavior The satellite should be created with a ghost record included.

chrispuuri commented 1 year ago

We ran into this one as well. This is a deal breaker from us using this feature at the moment.

DVAlexHiggs commented 1 year ago

We ran into this one as well. This is a deal breaker from us using this feature at the moment.

Hotfix on its way!

chrispuuri commented 1 year ago

Thank you sir!

On Thu, Dec 22, 2022 at 6:21 AM Alex Higgs @.***> wrote:

We ran into this one as well. This is a deal breaker from us using this feature at the moment.

Hotfix on its way!

— Reply to this email directly, view it on GitHub https://github.com/Datavault-UK/dbtvault/issues/173#issuecomment-1362897003, or unsubscribe https://github.com/notifications/unsubscribe-auth/AXEQEAZ6UWRP52FKOATHSP3WORPXVANCNFSM6AAAAAATDSHWLY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

--

http://www.hakkoda.io/

Chris Puuri

Healthcare & Data Engineering Lead

@.***

+1.619.884.5183 | Meet With Me https://meetings.hubspot.com/christopher-puuri

DVAlexHiggs commented 1 year ago

Released v0.9.2, this should be fixed :) Please re-open if you continue to have the same issue, or open a new issue if you experience any new issues. Thank you for this report!