search

databricks / dbt-databricks

A dbt adapter for Databricks.
https://databricks.com
Apache License 2.0
213 stars 114 forks source link

Materialized view refresh in progress is returned as an error #602

Open kdazzle opened 6 months ago

kdazzle commented 6 months ago

Describe the bug

A clear and concise description of what the bug is. What command did you run? What happened?

When the model for a materialized view is run, but that same model is already currently running in the background, the job returns as a failure. I would expect this to be a warning, since there was no actual error.

dbt run --select="tag:hourly-jobs" --target=prod

Steps To Reproduce

Run a long-running materialized view model that already exists (not a full-refresh). And while that is still running, run it again.

Expected behavior

I would expect it would return as a warning instead of a runtime error, since it's not entirely unexpected - it's just running a long-running job.

Screenshots and log output

If applicable, add screenshots or log output to help explain your problem.

Runtime Error in model latest_vehicle_location (models/marts/my_mart/my_materialized_view.sql)
  A refresh is already running in the background. You can monitor the previous refresh by visiting the following URL:

System information

The output of dbt --version:

23:00:28  Running with dbt=1.7.8
23:00:30  Registered adapter: databricks=1.7.8

The operating system you're using: dbt cloud

The output of python --version: dbt cloud

Additional context

Not a huge deal, but it creates noise that could hide real errors

benc-db commented 6 months ago

Can you validate if this error is still present in 1.8.0b1? I suspect that it is, but since that release has many changes to how MVs work, would really appreciate the extra info.

github-actions[bot] commented 3 weeks ago

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please remove the stale label or comment on the issue.