When running dbt pipelines each day, I'd like to automate when views are materialized and when they don't need to be using code version changes. If the code version has not changed then even if the view is part of my selection, Dagster will not add it to the dbt build command. If it has changed then it does get added.
Allowing users to specify a dbt selection that includes views yet offload whether or not to include them at runtime to Dagster will be hugely beneficial to users and selling point for orchestrating dbt with Dagster.
Ideas of implementation
The implementation would be something similar to asset checks. Within _get_subset_selection_for_context of the DbtResource.cli method views would have their current and previous code version checked. If they match, don't include those views in the selected_dbt_resources. If they don't, include them. If the view has never ran before, materialize it.
Additional information
Message from the maintainers
Impacted by this issue? Give it a 👍! We factor engagement into prioritization.
What's the use case?
When running dbt pipelines each day, I'd like to automate when views are materialized and when they don't need to be using code version changes. If the code version has not changed then even if the view is part of my selection, Dagster will not add it to the
dbt build
command. If it has changed then it does get added.Allowing users to specify a dbt selection that includes views yet offload whether or not to include them at runtime to Dagster will be hugely beneficial to users and selling point for orchestrating dbt with Dagster.
Ideas of implementation
The implementation would be something similar to asset checks. Within
_get_subset_selection_for_context
of theDbtResource.cli
method views would have their current and previous code version checked. If they match, don't include those views in theselected_dbt_resources
. If they don't, include them. If the view has never ran before, materialize it.Additional information
Message from the maintainers
Impacted by this issue? Give it a 👍! We factor engagement into prioritization.