search

microsoft / dbt-fabric

MIT License
79 stars 28 forks source link

Incremental models cannot full_refresh #179

Closed DezeriusCsanadLevay closed 4 months ago

DezeriusCsanadLevay commented 6 months ago

When I run an incremental model with --full-refresh it throws and error that table already exist. It doesn't matter which on_schema_change route I choose. I always have to manually drop the table to do a full refresh. image

Am I missing something or is this a bug?

prdpsvs commented 5 months ago

@DezeriusCsanadLevay , Could you please share a repro please? I will repro at my end and will fix this issue.

M4Al commented 5 months ago

I observe the same bug here :(

DezeriusCsanadLevay commented 5 months ago

@DezeriusCsanadLevay , Could you please share a repro please? I will repro at my end and will fix this issue.

I cannot share the exact repo (because it's private), but I recreated the issue with jaffleshop: image

Here's the repo: https://github.com/DezeriusCsanadLevay/jaffle-shop-classic.git

Actually you just have to define a model as incremental for the full-refresh to not work: {{ config( materialized='incremental', unique_key='customer_id', ) }}

byrontuckett commented 1 month ago

@prdpsvs Heads up that this issue is affecting dbt-synapse 1.8.1.

dbt-synapse 1.8.1 is requiring dbt-fabric==1.8.6. While the fixes are in dbt-fabric 1.8.7, which have been committed to the main branch, a release has not been cut yet to effect this change.

Would you kindly create a patch release so this fixes can be included for dnbt-synapse. Currently dbt--synapse users are unable to use --full-refresh with incremental models.