Datavault-UK / automate-dv

A free to use dbt package for creating and loading Data Vault 2.0 compliant Data Warehouses (powered by dbt, an open source data engineering tool, registered trademark of dbt Labs)
https://www.automate-dv.com
Apache License 2.0
513 stars 131 forks source link

[BUG] regression in 0.9.1 and later: Compilation error if hash key not defined #183

Closed thestonewell closed 1 year ago

thestonewell commented 1 year ago

Describe the bug While compiling macros with dbtvault version set to 0.9.3. If we don’t define the hash key in dbt_project.yml then it throws a compilation error.

Runtime Error

  Compilation Error in model v_stg_orders (models/data_vault/stage/v_stg_orders.sql)

    Required var 'hash' not found in config:

    Vars supplied to v_stg_orders = {

        "load_date": "1998-08-02",

        "tpch_size": 1

    }
    > in macro default__type_binary (macros/supporting/data_types/type_binary.sql)
    > called by macro type_binary (macros/supporting/data_types/type_binary.sql)

Macro leading to issue is default__type_binary.

{%- macro default__type_binary() -%}
    {%- if var('hash') | lower == 'md5' -%}
        BINARY(16)
    {%- elif var('hash') | lower == 'sha' -%}
        BINARY(32)
    {%- else -%}
        BINARY(16)
    {%- endif -%}
{%- endmacro -%}

In above macro hash key is being fetched from defined vars and if the key is missing than it leads to compilation error.

To fix the above issue we have modified the default__type_binary macro.

Modified macro

{%- macro default__type_binary() -%}
    {% set check_hash = var('hash', False) %}
    {%- if check_hash != False -%}
        {%- if var('hash') | lower == 'md5' -%}
            BINARY(16)
        {%- elif var('hash') | lower == 'sha' -%}
            BINARY(32)
        {%- endif -%}
    {%- else -%}
        BINARY(16)
    {%- endif -%}
{%- endmacro -%}

Environment

dbt version: 1.3 dbtvault version: 0.9.1 and later Database/Platform: Snowflake

To Reproduce see above

Expected behavior no regression

DVAlexHiggs commented 1 year ago

Thanks for the report. We'll get a fix and a regression test in for this!

johnoscott commented 1 year ago

I am experiencing this problem also (but using the workaround above). When will a new version be available with it fixed ?

DVAlexHiggs commented 1 year ago

Fixed and released in 0.9.4. Thank you. Please re-open if you continue to experience this issue