databrickslabs / dbx

🧱 Databricks CLI eXtensions - aka dbx is a CLI tool for development and advanced Databricks workflows management.
https://dbx.readthedocs.io
Other
439 stars 120 forks source link

Error "The job run failed because task dependency types are temporarily disabled" when using run_if option #827

Closed gmuciaccia-camlin closed 1 year ago

gmuciaccia-camlin commented 1 year ago

Expected Behavior

I am trying the newly released run_if option for tasks. I have created a workflow with one single leaf task that depends on multiple parent tasks and its run_if is set to AT_LEAST_ONE_SUCCESS.

The job definition is deployed with dbx and I would expect it to run without errors.

Current Behavior

The job fails when started with "The job run failed because task dependency types are temporarily disabled". However, the following procedure can fix it:

Steps to Reproduce (for bugs)

Just deploy with dbx a workflow which uses the run_if option for one of its tasks and try to run it.

Context

Related forum thread: https://community.databricks.com/t5/get-started-discussions/the-job-run-failed-because-task-dependency-types-are-temporarily/m-p/39166#M672

I'm not sure if this is a dbx bug or it's a problem with REST APIs.

Your Environment

gmuciaccia-camlin commented 1 year ago

It looks like the problem has been solved, everything works as expected now.