search

arnoN7 / dbt-incremental-stream

DBT Package reproducing dbt incremental materialization leveraging on Snowflake streams
MIT License
25 stars 4 forks source link

Ability to specify stream type #14

Closed dnabb closed 1 week ago

dnabb commented 3 weeks ago

Snowflake supports multiple types of streams.

In some cases, using a stream type other than the standard can be beneficial: as example, append-only stream are more performant than standard streams.

It would be nice if it were possible to specify the type of stream to be created by dbt, ideally as an optional argument to the stream_ref / stream_source macros.

arnoN7 commented 2 weeks ago

That's a great point! I'll look into adding it and will update here once I've made progress.

arnoN7 commented 2 weeks ago

As suggested, I've added an optional stream_type argument to the stream_ref and stream_source macros, allowing you to specify the desired stream type directly (STANDARD, APPEND_ONLY or INSERT_ONLY). The default value is STANDARD. I'd love to hear your feedback if you get a chance to try it out in your project!