snowplow / dbt-snowplow-utils

Snowplow utility functions to be used in conjunction with the snowplow-web dbt package.
Other
13 stars 6 forks source link

Grants not applied to models using snowplow_incremental materialization #135

Closed jakeberesford-palmetto closed 1 year ago

jakeberesford-palmetto commented 1 year ago

Describe the bug

Models using the snowplow_incremental materialization do not have dbt 'grants as config' applied.

Steps to reproduce

  1. Configure your dbt project with
    models:
      +grants:
        select: ANOTHER_ROLE # This is a different role than I am using to execute the model
  2. Create a new model with , materialized='snowplow_incremental' in the config
  3. Run the model
  4. Check the grants on the table in the warehouse
    show grants on table db.schema.my_table

Expected results

Configured grants are applied.

Actual results

No grants are applied.

Screenshots and log output

System information

The contents of your packages.yml file:

  - package: snowplow/snowplow_utils
    version: 0.13.2

I realize newer versions are available but the release notes do not mention this being fixed, and the materialization code does not contain a call to the apply_grants macro, so I believe this is still a bug.

Which database are you using dbt with?

The output of dbt --version:

dbt --version
Core:
  - installed: 1.4.6
  - latest:    1.5.2 - Update available!

  Your version of dbt-core is out of date!
  You can find instructions for upgrading here:
  https://docs.getdbt.com/docs/installation

Plugins:
  - snowflake: 1.4.1 - Update available!

The operating system you're using:

MacOS / Linux

The output of python --version:

Python 3.11.2

Additional context

https://github.com/dbt-labs/dbt-snowflake/blob/main/dbt/include/snowflake/macros/materializations/table.sql#L31

Are you interested in contributing towards the fix?

I'd love to, but I do not know how soon I will have time to contribute.

rlh1994 commented 1 year ago

Hi @jakeberesford-palmetto and thanks for reporting this issue. The latest version of the package (>=0.14.0) actually deprecates this materialization entirely - instead we add an extra step in the dbt materialization call rather than forking and using our own. This means that basically every feature DBT has added to the incremental materialization since we forked it a few years ago is available; this should include the call to the apply_grants macro, and also includes things like on_schema_change which we also didn't support before. We didn't list every feature as it is just any feature that DBT supports. In version 0.15.0 we expect to remove the custom materialization entirely, so it's unlikely we will add features to previous versions, as the difficulty in bringing our custom materialization in line with the latest dbt core one was one of the reasons we took this direction.

I don't see any reason that this would not working in any version 0.14.0 or greater (using the standard incremental materialization and snowplow_optimize = true ), but will attempt to confirm tomorrow if you are not able to before then.

jakeberesford-palmetto commented 1 year ago

Thanks @rlh1994 - I'll give v0.14 a shot with the new snowplow_optimize config

jakeberesford-palmetto commented 1 year ago

Looks like that works for me, this issue can be closed out! Thanks again @rlh1994

rlh1994 commented 1 year ago

Great to hear!