Open thomasdkelly opened 8 months ago
@dbeatty10 Would you accept a PR for this? Just got hit by this again. We had a dataset that had a default partition expiration of 7 days. Couldn't understand why incremental model's with a partition expiration of 3 years were failing to insert any data.
Is this a new bug in dbt-bigquery?
Current Behavior
In our GCP project, we set a default partition expiry of 3 years. However, we have one model that we want to go back to 5 years. This model has the following config:
When attempting to backfill this model for 2021-01-01 (more than 3 years ago), the model is running but the resulting partition is empty. The query returns data but the temp table uses the default project partition expiry of 3 years (instead of the 5 years specified by the model).
Expected Behavior
I expect the temp table to have the same expiry of the model it wishes to merge into.
Steps To Reproduce
In your gcp project, attempt to run the model for a date within your models config partition expiry (but greater than your projects default expiry). For example this project has a default expiry of 3 years, which means this partition for 2021-01-01 will is outside of the temp table expiry. (but within the tom_demo model's expiry).
Relevant log output
No response
Environment
Additional Context
No response