Added code to support Run Job, JAR and SQL task types. Run Job task can trigger another job by its job_name, JAR task will trigger by providing jar file as library and its main class and SQL tasks will be useful to execute SQL objects such as queries, legacy dashboards, files and alerts
Databricks now supports creating a "Run Job" task, which can trigger another job by its job_id, its good to have this feature in brickflow.
Databricks now supports creating a "JAR" task. Using the JAR task, you can ensure fast and reliable installation of Java or Scala code in your Databricks jobs.
Databricks now supports creating a "SQL" task. Using the SQL task, SQL tasks will be useful to execute SQL objects such as queries, legacy dashboards, files and alerts
How Has This Been Tested?
Unit tests + deployed test workflow using wheel file
Screenshots (if appropriate):
All test cases passed:
Tested using brickflow-bundles-test:
Test workflow with a JAR task created using whl file:
Test workflow with a Run Job task created using whl file:
Test workflow with a SQL job task created using whl file:
Types of changes
[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
Checklist:
[x] My code follows the code style of this project.
[x] My change requires a change to the documentation.
Description
Added code to support
Run Job, JAR and SQL task
types. Run Job task can trigger another job by its job_name, JAR task will trigger by providing jar file as library and its main class and SQL tasks will be useful to execute SQL objects such as queries, legacy dashboards, files and alertsRelated Issue
Link to the issue: JAR #49 Link to the issue: RunJob #55 Link to the issue: SQL #119
Motivation and Context
How Has This Been Tested?
Unit tests + deployed test workflow using wheel file
Screenshots (if appropriate):
All test cases passed:
Tested using brickflow-bundles-test:
Types of changes
Checklist: