Microbatch performance is underwhelming for dbt-snowflake! A couple underlying reasons:
1) Microbatch largely inherits from delete+insert strategy in snowflake, which has been optimized to use a temp view instead of table. However, microbatch is still using a temp table! More details: https://docs.getdbt.com/reference/resource-configs/snowflake-configs#temporary-tables
Note that it is always safe to do so for microbatch since there is no unique_key necessary
2) The using clause is unnecessary in the microbatch delete statement, because microbatch does not require a merge key and the where clause is sufficient for a safe deletion.
Solution
1) Apply temp view optimization to microbatch. Was just a matter of extending a couple if statements that only checked for insert+delete previously
2) Remove extraneous using clause
[x] I have run this code in development and it appears to resolve the stated issue
[x] This PR includes tests, or tests are not required/relevant for this PR
[x] This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc) or this PR has already received feedback and approval from Product or DX
Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the dbt-snowflake contributing guide.
resolves #1228 docs dbt-labs/docs.getdbt.com/# N/A
Problem
Microbatch performance is underwhelming for dbt-snowflake! A couple underlying reasons: 1) Microbatch largely inherits from delete+insert strategy in snowflake, which has been optimized to use a temp view instead of table. However, microbatch is still using a temp table! More details: https://docs.getdbt.com/reference/resource-configs/snowflake-configs#temporary-tables
unique_key
necessary 2) Theusing
clause is unnecessary in the microbatchdelete
statement, because microbatch does not require a merge key and thewhere
clause is sufficient for a safe deletion.Solution
1) Apply temp view optimization to microbatch. Was just a matter of extending a couple if statements that only checked for insert+delete previously 2) Remove extraneous
using
clauseChecklist