search

dbt-labs / dbt-snowflake

dbt-snowflake contains all of the code enabling dbt to work with Snowflake
https://getdbt.com
Apache License 2.0
296 stars 176 forks source link

[Bug] dbt snowflake no longer works for microbatch #1248

Open sarahjryan opened 1 week ago

sarahjryan commented 1 week ago

Is this a new bug in dbt-snowflake?

Current Behavior

When calling dbt build on incremental models configured with strategy microbatch I'm now (as of afternoon 12th Nov 24) getting this error:

Compilation Error in model xxx The flag require_batched_execution_for_custom_microbatch_strategy has not be registered.

in macro materialization_incremental_snowflake (macros\materializations\incremental.sql) called by model xxx

Checking the dbt docs a change has been made so DBT_EXPERMINENTAL_MICROBATCH no longer affects runtime functionality. The article refers to the flag specified in the error and says false is default, no need to specify. Tried setting to true jsut to check & no microbatching occurs.

https://docs.getdbt.com/reference/global-configs/behavior-changes#custom-microbatch-strategy

Expected Behavior

The microbatching works w/o error as it did prior to the change, when the env var was used

Steps To Reproduce

Configure an incremental as microbatch and run for more than one batch (the first batch seems to work w/o issue)

Relevant log output

No response

Environment

- OS: windows 11
- Python: 3.10.9
- dbt-snowflake==1.9.0b1
- dbt-core==1.9.0b2

Additional Context

No response

QMalcolm commented 3 days ago

Hi @sarahjryan, are you stilling running into this issue? If you are, can you run a pip freeze | grep dbt-adapters and post the output?

To jump a head as to why I'm asking for the dbt-adapters version, we released a bad version of dbt-adapters, 1.11.0, which caused this issue. We've since yanked that version from PyPI. Moving to dbt-adapters 1.9.0 (which interestingly was released after 1.11.0 🫠) should resolve the issue.