search

databricks / dbt-databricks

A dbt adapter for Databricks.
https://databricks.com
Apache License 2.0
195 stars 104 forks source link

Model level retry #717

Open gaoshihang opened 4 days ago

gaoshihang commented 4 days ago

Describe the feature

Hi @benc-db , Does DBT currently support model level retry? I didn't find information in DBT website. But I think this can be a useful feature, because sometimes the error is fortuitous. So if we can have model level retry, we can config a retry number, and DBT can help us re-submit the model to Databricks.

Describe alternatives you've considered

Just fail-fast, then use "dbt retry" to rerun all models.

Additional context

Please include any other relevant context here.

Who will this benefit?

What kind of use case will this feature be useful for? Please be specific and provide examples, this will help us prioritize properly.

Are you interested in contributing this feature?

Yes, First can you help confirm currently we don't have this.

benc-db commented 2 days ago

There are a number of retries that happen before you even see an error, so it kind of depends on what sort of error that's surfacing. Do you have an example of a transient error that would/could succeed on retry? For many transient errors, the retries are just invisible to the user.