search

databricks / cli

Databricks CLI
Other
132 stars 50 forks source link

Add prompt when a pipeline recreation or deletion happens #1672

Closed shreyas-goenka closed 3 weeks ago

shreyas-goenka commented 1 month ago

Changes

DLT pipeline recreations are destructive. They can lead to lost history of previous updates, outage of the tables temporarily and are potentially computationally expensive. Thus we make a breaking change where a prompt is shown to the user if there configuration changes will lead to a DLT recreation.

Users can skip the prompt by specifying the --auto-approve flag.

Tests

Manually, and new unit and integration tests.

➜  bundle-playground-3 cli bundle deploy
Uploading bundle files to /Users/63ec021d-b0c6-49c0-93a0-5123953a1cb2/.bundle/test/development/files...
The following DLT pipelines will be recreated. Underlying tables will be unavailable for a transient period until the newly recreated pipelines are run once successfully. History of previous pipeline update runs will be lost because of recreation:
  recreate pipeline foo

Would you like to proceed? [y/n]: n
Deployment cancelled!
shreyas-goenka commented 1 month ago

Triggered nighties on this PR...

shreyas-goenka commented 4 weeks ago

Trigger nightlies again...

shreyas-goenka commented 3 weeks ago

Nightlies are green on this PR.