databrickslabs / dbx

🧱 Databricks CLI eXtensions - aka dbx is a CLI tool for development and advanced Databricks workflows management.
https://dbx.readthedocs.io
Other
439 stars 120 forks source link

[docs] DLT example deployment file missing "default" #805

Open BGerwe opened 1 year ago

BGerwe commented 1 year ago

Expected Behavior

Using the DLT pipeline deployment file as a template, I should be able to fill in appropriate values and deploy the pipeline successfully.

Current Behavior

Trying to deploy the pipeline results in AttributeError: 'list' object has no attribute 'get'.

Steps to Reproduce (for bugs)

  1. Create a minimal deployment YAML based on the example deployment file. This looks something like:

conf/dlt_deployment.yml

environments:
  workflows:
    - name: "test_DLT_pipeline"
      workflow_type: "pipeline" 
      storage: "dbfs:/user/location"
      configuration:
        "property1": "value"
      clusters:
        - label: "some-label"
          autoscale:
            min_workers: 1
            max_workers: 2
            mode: "legacy" 
      libraries:
        - notebook:
            path: "/Repos/user/path_to_notebook"
      target: "some_target_db"
  1. Deploy the pipeline: dbx deploy --deployment-file conf/dlt_deployment.yml

Context

This is fixed by adding a default: level after environments:

environments:
  default:
    workflows:
      - name: "test_DLT_pipeline"
 ...

Your Environment

python 3.9.15