If a scheduled refresh is already in progress when you request a refresh, you get an exception. Exception handling during a materialization is gnarly, so instead, I ensure that any in progress pipelines are completed before we do any further processing of an MV/ST.
This PR also removes polling for completion of a create or refresh kicked off by dbt, as those operations are now synchronous by default. Since I already have the pipeline id when I do need to check the refresh state, I can get rid of the API call we were using to get the pipeline id, which fixes another bug: refresh for MV/ST wasn't working in dbt-databricks if you needed to escape your relation name (since we had to strip off the escaping to call the API).
Checklist
[x] I have run this code in development and it appears to resolve the stated issue
[x] This PR includes tests, or tests are not required/relevant for this PR
[x] I have updated the CHANGELOG.md and added information about my change to the "dbt-databricks next" section.
Description
If a scheduled refresh is already in progress when you request a refresh, you get an exception. Exception handling during a materialization is gnarly, so instead, I ensure that any in progress pipelines are completed before we do any further processing of an MV/ST.
This PR also removes polling for completion of a create or refresh kicked off by dbt, as those operations are now synchronous by default. Since I already have the pipeline id when I do need to check the refresh state, I can get rid of the API call we were using to get the pipeline id, which fixes another bug: refresh for MV/ST wasn't working in dbt-databricks if you needed to escape your relation name (since we had to strip off the escaping to call the API).
Checklist
CHANGELOG.mdand added information about my change to the "dbt-databricks next" section.