Datavault-UK / automate-dv

A free to use dbt package for creating and loading Data Vault 2.0 compliant Data Warehouses (powered by dbt, an open source data engineering tool, registered trademark of dbt Labs)
https://www.automate-dv.com
Apache License 2.0
511 stars 131 forks source link

Add grant execution to pit and bridge materialization #242

Open cd-philipp opened 3 months ago

cd-philipp commented 3 months ago

_Relatedissue

Describe the bug When using the materializations pit_incremental or bridge_incremental, Grant statements defined in the _dbtproject or in models directly are not being executed at all, not even for models with other materializations like the official materializations provided by dbt.

We are aware that the PIT and BRDIGE macros have been deprecated in the latest version of AutomateDV, however, it would be great if this can still be fixed to ensure users can still make use of those macros until alternatives are provided.

Environment

dbt version: 1.7.13 automate_dv version: 0.11.0 Database/Platform: Snowflake

To Reproduce Steps to reproduce the behavior: [Option 1] Test in production

  1. Add a grant statement for an existing database role e.g. in the _dbtproject for a directory or model that is materialized as pit_incremental or bridge_incremental
  2. Merge the changes and run the PROD pipeline
  3. Check the grants on the respective tables in Snowflake by running show grants on table <database_name>.<schema_name>.<table_name>;
  4. The database role that has actually been assigned in dbt has not been granted

[Option 2] Test locally

  1. Add a grant statement for a database role that does not exist e.g. in the _dbtproject for a directory or model that is materialized as pit_incremental or bridge_incremental
  2. Execute a dbt_run for the respective model
  3. Then dbt run will be successful (although it should fail)

Expected behavior [Option 1] The database role that has been assigned in dbt should be granted to the respective table(s)

[Option 2] The dbt run should fail and a SQL compilation error should be thrown stating that "Role '' does not exist or not authorized", given that the role name does actually not exist.

Screenshots dbt_project config for Business Vault dbt_project_config_business_vault

Additional context In version 1.2 dbt has added the execution of Grant configs to their materialization macros (Commit). However, it seems like this hasn´t been added to the AutomateDV materialization macros for PIT and BRIDGE tables.