Open antgei opened 5 months ago
Suggest for patch to make it work:
diff --git a/dbx/api/adjuster/adjuster.py b/dbx/api/adjuster/adjuster.py
index 4ae4301..8ad5f2c 100644
--- a/dbx/api/adjuster/adjuster.py
+++ b/dbx/api/adjuster/adjuster.py
@@ -85,7 +85,8 @@ class PropertyAdjuster(
for element, _, __ in self.traverse(workflows):
if isinstance(element, (V2dot0Workflow, JobTaskSettings)):
- self._preprocess_libraries(element, additional_libraries)
+ if element.sql_task is None: # SQL Task does not support dependent libraries
+ self._preprocess_libraries(element, additional_libraries)
def _new_cluster_handler(self, element: NewCluster):
# driver_instance_pool_name -> driver_instance_pool_id
diff --git a/dbx/models/workflow/v2dot1/task.py b/dbx/models/workflow/v2dot1/task.py
index e063bb4..0ba09c8 100644
--- a/dbx/models/workflow/v2dot1/task.py
+++ b/dbx/models/workflow/v2dot1/task.py
@@ -42,7 +42,8 @@ class SqlTaskAlert(FlexibleModel):
class SqlFile(FlexibleModel):
- file: str
+ path: str
+ source: str
class SqlTask(FlexibleModel):
Is there any update on this? facing same issue
Expected Behavior
Current Behavior
Problem 1
ValidationError: 1 validation error for Deployment workflows -> 0 -> Workflow -> tasks -> 2 -> sql_task -> file -> file field required (type=value_error.missing)
This is wrong according to current API, expects sql_task -> file -> path: https://docs.databricks.com/api/workspace/jobs/create
Problem 2
The scond problem is that "libraries" including the .whl are appended to all tasks, this is not allowed for sql_task
Steps to Reproduce (for bugs)
dbx deploy ...
Context
Your Environment