Open tinolyuu opened 5 days ago
@tinolyuu Would you be able to explain your use case in which you are dropping tables outside of dbt as dbt is executing its metadata queries? This seems like an anti-pattern for what we expect
@tinolyuu Would you be able to explain your use case in which you are dropping tables outside of dbt as dbt is executing its metadata queries? This seems like an anti-pattern for what we expect
We may have multiple jobs running at the same time, so a dbt process may accidentally perform drop table query while another one running show table extended in database like '*'
.
Is this a new bug in dbt-spark?
Current Behavior
Got the following errors when running models. It happens when some tables in the same database get deleted during dbt running the
show table extended in database like '*'
query.Expected Behavior
Try more attempts before failing the query only if it gets
TABLE_OR_VIEW_NOT_FOUND
error.Steps To Reproduce
Run a dbt model, instantly delete a table from the database when it outputs
show table extended in database like '*'
.Relevant log output
No response
Environment
Additional Context
In the previous version, we modified list_relations_without_caching to try some attempts before failing the query.
But this workaround doesn't work now as more exceptions are being caught as
DbtRuntimeError
in the later version. We couldn't catch the exactTABLE_OR_VIEW_NOT_FOUND
error.