dbt-labs / dbt-core

dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.
https://getdbt.com
Apache License 2.0
9.74k stars 1.61k forks source link

[Feature] Add a flag to clone to skip selected models that don't exist #10841

Open Jacob-Bishop opened 4 hours ago

Jacob-Bishop commented 4 hours ago

Is this your first time submitting a feature request?

Describe the feature

There should be an option for the dbt clone command to skip models that don't exist in the source schema instead of failing.

Describe alternatives you've considered

This can be done manually via a query run against the catalog to find out what tables exist in the source with the results of that used to construct the select statement in the clone command.

Who will this benefit?

My use case for this is a slim-ci variant where we clone data from production to our CI schema and then only rebuild modified tables. However, this breaks if a new model has been merged to main since the last time our production dbt job has run (since clone tries to copy a table that doesn't yet exist).

Are you interested in contributing this feature?

No response

Anything else?

Jacob-Bishop commented 4 hours ago

Discussed with @ChenyuLInx at coalesce