How to deploy scheduled jobs to a Databricks from Azure DevOps CICD assuming there is an exported json configuration from Databricks -> Jobs -> scheduled job json

[chengyu-liu-cs]

[Enter feedback here]

Hi, In Azure DevOps, there is a way to deploy notebook to Databricks workpsace. Similarly, is there a way to deploy Databricks scheduled jobs from Azure DevOps using CICD. In this case configuration json file which can be exported from Databricks workspace (see example below).

{ "existing_cluster_id": "existing_cluster_id_123456", "notebook_task": { "notebook_path": "/Users/XYZ/example_notebook.py" }, "email_notifications": { "on_start": [ "XYZ@example.com" ], "on_success": [ "XYZ@example.com" ], "on_failure": [ "XYZ@example.com" ] }, "name": "example_scheduler", "schedule": { "quartz_cron_expression": "0 10/30 12,13,14 ? 2021", "timezone_id": "Europe/Kiev", "pause_status": "UNPAUSED" }, "max_concurrent_runs": 1 }

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: 2e3a7fd6-a11f-0e9f-9747-7574d1096810
• Version Independent ID: 7fdd2591-162b-a0a7-8ff7-b94c93829804
• Content: Jobs - Azure Databricks
• Content Source: databricks/jobs.md
• Service: azure-databricks
• GitHub Login: @mssaperla
• Microsoft Alias: saperla

[CHEEKATLAPRADEEP-MSFT]

@chengyu-liu-cs Since this issue is not related directly to the doc, I would recommend you to create a thread on the forums - Microsoft Q&A or Stack Overflow Once you post your issue on forums, it will have visibility across the community which is a better suited audience for such types of issues.

[chengyu-liu-cs]

@CHEEKATLAPRADEEP-MSFT I am not sure why this is not directly related. Both Azure Databricks and Azure DevOps are Microsoft services and we are paying for the services. But I did not find documentation for connecting these two services. I dont understand why I should ask in Stack Overflow.

Should I rename title, such as "Missing doc for connecting azure devops and databricks scheduled jobs"?

[CHEEKATLAPRADEEP-MSFT]

@chengyu-liu-cs Thanks for the clarification. I have assigned the issue to the content author to evaluate and create a new documentation for connecting azure Devops and databricks scheduled jobs.

[kateglee-db]

Thanks for your dedication to our documentation. Unfortunately, we have been unable to address your issue and apologize for the delayed response. We are closing this issue, but if you feel that it's still a concern, please let us know directly at doc-feedback@databricks.com.

please-close

[prmerger-automator[bot]]

Invalid command: '#please-close'. Only Microsoft employees can use this command.

[JasonWHowell]

please-close