entechlog / dbt-snow-mask

This repository contains source code for dbt package dbt_snow_mask.
https://hub.getdbt.com/entechlog/dbt_snow_mask/latest/
GNU General Public License v3.0
60 stars 24 forks source link

apply policy to all models #43

Open mbritton-neuroflow opened 2 years ago

mbritton-neuroflow commented 2 years ago

Is there a reason that dbt run-operation apply_masking_policy --args '{"resource_type": "sources"}' works for sources but not models? And that models need to have masking policies applied as a post-hook or on-run-end? I am able to run the above command with a resource_type of models but it does nothing.

It would be very helpful to have this feature to mitigate the difficulty of unapplying and reapplying masking policies when they change, because running all models can take a long time, but applying masking policies should be very quick.

dweaver33 commented 2 years ago

Seems you're having the same issue as me here #42

entechlog commented 2 years ago

dbt run-operation apply_masking_policy --args '{"resource_type": "models"}'

This was never a supported feature of this package\macro. apply_masking_policy happens after each model run as a post hook, adding this support may need further development

Chobsz commented 2 years ago

Think my PullRequest might cover this issue, check out the readme-edits to see the added functionality. Pull41

mbritton-neuroflow commented 2 years ago

Yes, that looks like it meets intent! Thanks!

lilitangsonos commented 1 year ago

@entechlog In the instructions it says you can apply the masking policy by applying dbt run -- model <model-name> which it doesn't seem like this works. It needs to be done in the post hook as you also mentioned in #42. Should probably remove that instruction unless I'm misunderstanding here...

mgeerken commented 1 year ago

Is it an ongoing discussion? 'Cause in the readme it still claims an on-run-end would work, but the macro is checking on the resource_type which is "macro" on an on-run-end.

Alternatively, you can also apply the masking policies by specifying below post-hook OR on-run-end to dbt_project.yml

{% set model_resource_type = model.resource_type | string %} {% if model_resource_type|lower in ["model", "snapshot"] %}