Datavault-UK / automate-dv

A free to use dbt package for creating and loading Data Vault 2.0 compliant Data Warehouses (powered by dbt, an open source data engineering tool, registered trademark of dbt Labs)
https://www.automate-dv.com
Apache License 2.0
511 stars 131 forks source link

[BUG] Staging includes duplicate or missing columns in certain scenarios #21

Closed DVAlexHiggs closed 3 years ago

DVAlexHiggs commented 3 years ago
  1. When using the stage macro, if hashed columns have the same name as a source column, a DUPLICATE COLUMN SQL error will be raised.

The expected behaviour is that the hashed_column definition will take precedence over the original source column, and the source column will not be present in the final stage table.

  1. Additionally, if include_source_columns = true and a derived_columnis defined which derives its value from a source column, then the source column will not be included in the final stage layer.

The expected behaviour is that both the original column and the new aliased column will be present in the final stage layer.

We plan to fix this for the next release.