Update BQ deduplicate macro to support partition pruning downstream - Githubissues

dbt-labs / dbt-utils

Utility functions for dbt projects.

https://hub.getdbt.com/dbt-labs/dbt_utils/latest/

Apache License 2.0

1.38k stars 499 forks source link

Update BQ deduplicate macro to support partition pruning downstream #929

Open austinclooney opened 4 months ago

austinclooney commented 4 months ago

resolves https://github.com/dbt-labs/dbt-utils/issues/928

Problem

The current BQ deduplicate macro will not work for a view as it doesn't allow partition pruning downstream of the macro.

Solution

Add an an optional parameter that explicitly selects the partition by columns outside of the array_agg

Checklist

[x] This code is associated with an issue which has been triaged and accepted for development.
[ ] I have read the contributing guide and understand what's expected of me
[ ] I have run this code in development and it appears to resolve the stated issue
[ ] This PR includes tests, or tests are not required/relevant for this PR
[ ] I have updated the README.md (if applicable)