databricks / bundle-examples

Examples of Databricks Asset Bundles
Other
57 stars 23 forks source link

Add example of a job with notebook tasks #2

Closed pietern closed 11 months ago

pietern commented 11 months ago

Simple example to showcase how to return a value from a notebook and work with it from databricks bundle run.

pietern commented 11 months ago

Text output:

% databricks bundle run job_with_notebook_tasks 
Run URL: https://...

2023-08-11 09:08:55 "Demonstrate programmatically retrieving notebook output" TERMINATED SUCCESS 
Output:
=======
Task task1:
Widget parameter 'hello' contains: world
=======
Task task2:
Widget parameter 'hello' contains: universe

JSON output:

% databricks bundle run job_with_notebook_tasks --output json
Run URL: https://...

2023-08-11 12:20:19 "Demonstrate programmatically retrieving notebook output" TERMINATED SUCCESS 
{
  "task_outputs": [
    {
      "TaskKey": "",
      "Output": null,
      "EndTime": 0
    },
    {
      "TaskKey": "",
      "Output": null,
      "EndTime": 0
    },
    {
      "TaskKey": "task1",
      "Output": {
        "result": "Widget parameter 'hello' contains: world"
      },
      "EndTime": 1691749211504
    },
    {
      "TaskKey": "task2",
      "Output": {
        "result": "Widget parameter 'hello' contains: universe"
      },
      "EndTime": 1691749211520
    }
  ]
}