search

bruno-szdl / dbt-ci-cd

94 stars 18 forks source link

Improve testing of incremental models #23

Open benedikt-buchert opened 3 months ago

benedikt-buchert commented 3 months ago

When testing incremental models in ci it may happen they succeed even though they fail on prod this can be worked around as described here:

https://docs.getdbt.com/best-practices/clone-incremental-models

dbt clone --select state:modified+,config.materialized:incremental,state:old

Then

dbt build --select state:modified+

benedikt-buchert commented 3 months ago

To make this even more perfect a full refresh can be tried if the first one fails.

If full refresh succeeds a warning comment can be added by the workflow that there is a breaking schema change.

This can be adjusted to also run full refresh for these models in prod too.

https://docs.github.com/en/actions/learn-github-actions/variables#defining-configuration-variables-for-multiple-workflows

benedikt-buchert commented 1 week ago

PR for feature 1: https://github.com/bruno-szdl/dbt-ci-cd/pull/24