search

Datavault-UK / automate-dv

A free to use dbt package for creating and loading Data Vault 2.0 compliant Data Warehouses (powered by dbt, an open source data engineering tool, registered trademark of dbt Labs)
https://www.automate-dv.com
Apache License 2.0
513 stars 131 forks source link

[BUG] vault_insert_by_period does not work for hourly loads on BQ #178

Closed balmasi closed 1 year ago

balmasi commented 1 year ago

Describe the bug

When using period='HOUR' on the vault_insert_by_period materialization, the load fails.

Environment

dbt version: 1.3 dbtvault version: 0.9.2 Database/Platform: Bigquery

To Reproduce Steps to reproduce the behavior:

  1. Use the vault_insert_by_period materialization on a satellite with the HOUR period

Expected behavior

This is happening cause in BQ we typically use the timestamp type for datetime metadata and DATE_ADD used in dbtvault only works down to the DAY grain.

For the correct behaviour, you'd need to cast to a TIMESTAMP and use TIMESTAMP_ADD

Workaround

Use the vault_insert_by_rank materization instead.

DVAlexHiggs commented 1 year ago

Hi, thanks for your report! We'll look into this on the new year. Do you mean dbtvault v0.9.2? 0.9.4 doesn't currently exist.

balmasi commented 1 year ago

Hi, thanks for your report! We'll look into this on the new year. Do you mean dbtvault v0.9.2? 0.9.4 doesn't currently exist.

Yup. my bad.

Happy holidays!

DVAlexHiggs commented 1 year ago

Hi! We've added some fixes for this in 0.9.5. Further improvements coming in 0.9.6. Please re-open if they continue. Thanks :)