Datavault-UK / automate-dv

A free to use dbt package for creating and loading Data Vault 2.0 compliant Data Warehouses (powered by dbt, an open source data engineering tool, registered trademark of dbt Labs)
https://www.automate-dv.com
Apache License 2.0
511 stars 131 forks source link

Custom hash characters (concat_string and null_placeholder_string) #67

Closed koillinengit closed 3 years ago

koillinengit commented 3 years ago

Is your feature request related to a problem? Please describe. Different customers may have different hashing concatenation string (default ||) and null placeholder string (^^). So if you are extending old solution with dbtvault solution the hashes would not match. So it would be nice to be able to change those values in dbt_project.yml like this:

vars: hash: "MD5" concat_string: "||" null_placeholder_string: "^^"

Describe the solution you'd like I would like that local concat_string and null_placeholder_string variables would be changed as project variables in hash.sql macro.

So this code should be changed

{%- set concat_string = "||" -%} {%- set null_placeholder_string = "^^" -%}

like this (same way as the hash variable is already)

{%- set concat_string = var('concat_string','||') -%} {%- set null_placeholder_string = var('null_placeholder_string','^^') -%}

DVAlexHiggs commented 3 years ago

Thanks for this! This is actually already in our backlog and something we want to implement, but just haven't gotten around to yet. We'll keep this thread updates as and when things progress

koillinengit commented 3 years ago

Anxiously waiting for this :) I have done it already in my own custom version and it works nicely as I described. It would be better if it would be in official version, so I could get rid of custom one.

DVAlexHiggs commented 3 years ago

Implemented in #70. This will be in dbtvault 0.7.8. Will close once released.

koillinengit commented 3 years ago

Jei, excellent!!! Thanks

DVAlexHiggs commented 3 years ago

v0.7.8 Released