Datavault-UK / automate-dv

A free to use dbt package for creating and loading Data Vault 2.0 compliant Data Warehouses (powered by dbt, an open source data engineering tool, registered trademark of dbt Labs)
https://www.automate-dv.com
Apache License 2.0
478 stars 114 forks source link

[BUG] 'dict object' has no attribute 'replace_placeholder_with_filter' in vault_insert_by_period_materialization on DBT/Snowflake #165

Closed hrvoje-intuita closed 1 year ago

hrvoje-intuita commented 1 year ago

Describe the bug I'm receiving

Compilation Error in macro materialization_vault_insert_by_period_default (macros/materializations/vault_insert_by_period_materialization.sql)
  'dict object' has no attribute 'replace_placeholder_with_filter'

in vault_insert_by_period_materializationwhile running on DBT in cloud with Snowflake.

Environment dbt version: 1.2 dbtvault version: 0.9.0 Database/Platform: DBT Cloud

To Reproduce This is sample model that is generating error:

{{
  config(
    materialized = "vault_insert_by_period",
    period = "hour",
    timestamp_field = "SALEHOUR",
    start_date = backfill_startdate(),
    stop_date = backfill_enddate())
}}

SELECT market::STRING as market,
       f.value:billId::STRING as billId,
       ...other fileds...
       SALEHOUR     
FROM
(SELECT DISTINCT 
        ...other attributes...
       SALEHOUR,
       tradeJSON:trade_json.trades::VARIANT as trades       
From (select tradeJSON,DATETIME as SALEHOUR from "{{var("source_database")}}"."{{var("uat_source_schema")}}"."{{var("uat_source_demand_table")}}")
where tradeJSON:trade_json.trades[0] is not null and
       __PERIOD_FILTER__ ), TABLE(flatten(trades)) f 
GROUP BY 
    ...other attributes
       SALEHOUR

Backfill start and end dates are created with

{% macro backfill_enddate() %}
        {{var("backfill_enddate")}}
{% endmacro %}

{% macro backfill_startdate() %}
        {{var("backfill_startdate")}}
{% endmacro %}

Expected behavior insert_by_periodversion by dbt currently works well except for error in last hour. It iterates by hour and reports number of records inserted by each hour. It only fails on last hour.

DVAlexHiggs commented 1 year ago

Closing as this should now be fixed in 0.9.1. Please re-open if not! Thanks for reporting 😄