The insert_by_period materialization is very useful when populating large amounts of historical data. It enables the efficient use of indexes and allows work to be chunked into smaller pieces.
Having started transitioning our work to dbt recently, I have a working implementation of the insert_by_period materialization that's currently running in production. There are a few additional features I've implemented to allow some developer flexibility. I think it is now stable enough to contribute externally.
The
insert_by_periodmaterialization is very useful when populating large amounts of historical data. It enables the efficient use of indexes and allows work to be chunked into smaller pieces.Currently, this materialization is disabled for TSQL.
Having started transitioning our work to dbt recently, I have a working implementation of the insert_by_period materialization that's currently running in production. There are a few additional features I've implemented to allow some developer flexibility. I think it is now stable enough to contribute externally.
The implementation is based on Alex Higg's implementation for snowflake.
cc: @alittlesliceoftom