databricks / cli

Databricks CLI
Other
148 stars 56 forks source link

[BUG] `databricks labs upgrade` fails when dependencies used in install script have breaking changes #1623

Closed JCZuurmond closed 3 months ago

JCZuurmond commented 3 months ago

Describe the issue

The databricks labs upgrade command fails when dependencies used in install script have breaking changes.

See the following issues for ucx reporting problems related to this:

Steps to reproduce the behavior

Please list the steps required to reproduce the issue, for example:

  1. Install older ucx version: databricks labs install ucx@v0.23.1
  2. Upgrade to latest ucx version: databricks labs upgrade ucx

Expected Behavior

Upgrade should not fail

Actual Behavior

Upgrade fails due to missing dependencies.

OS and CLI version

Please include the version of the CLI (eg: v0.1.2) and the operating system (eg: windows). You can run databricks --version to get the version of your Databricks CLI

Is this a regression?

No

Debug Logs

Debug log for databricks --debug labs upgrade ucx

~/github/databrickslabs/ucx fix/install-deployment *2 ❯ databricks --debug labs upgrade ucx                                                                                                                      ▼ Py ucx 11:59:40
11:59:43  INFO start pid=46827 version=0.216.0 args="databricks, --debug, labs, upgrade, ucx"
11:59:43 DEBUG Latest ucx version is: v0.29.0 pid=46827
11:59:43 DEBUG Reading labs.yml@v0.29.0 from databrickslabs/ucx pid=46827
⣾ Cleaning up previous installation if necessary11:59:43 DEBUG Downloading zipball for v0.29.0 from databrickslabs/ucx pid=46827
⣷ Downloading and unpacking zipball for v0.29.011:59:45 DEBUG Unpacking zipball to: /Users/user/.databricks/labs/ucx/lib pid=46827
11:59:45 DEBUG Writing installed version info to: /Users/user/.databricks/labs/ucx/state/version.json pid=46827
11:59:45 DEBUG Launching installer script src/databricks/labs/ucx/install.py in /Users/user/.databricks/labs/ucx/lib pid=46827
11:59:45 DEBUG Loading login configuration from: /Users/user/.databricks/labs/ucx/config/login.json pid=46827
11:59:45 DEBUG Using workspace-level login profile: DEFAULT pid=46827
11:59:45 DEBUG Loading DEFAULT profile from /Users/user/.databrickscfg pid=46827 sdk=true
11:59:45 DEBUG Resolved login: Config: host=adb-REDACTED.azuredatabricks.net, cluster_id=REDACTED, account_id=REDACTED, profile=DEFAULT, config_file=/user/.databrickscfg. Env: DATABRICKS_HOST, DATABRICKS_CLUSTER_ID, DATABRICKS_ACCOUNT_ID pid=46827 sdk=true
11:59:45 DEBUG Passing down environment variables: DATABRICKS_ACCOUNT_ID, DATABRICKS_AUTH_TYPE, DATABRICKS_HOST, DATABRICKS_CLUSTER_ID pid=46827
11:59:45 DEBUG starting: /Users/user/.databricks/labs/ucx/state/venv/bin/python3 src/databricks/labs/ucx/install.py pid=46827
Traceback (most recent call last):
  File "/Users/user/.databricks/labs/ucx/lib/src/databricks/labs/ucx/install.py", line 27, in <module>
    from databricks.labs.lsql.dashboards import DashboardMetadata, Dashboards
ModuleNotFoundError: No module named 'databricks.labs.lsql.dashboards'
Error: installer: exit status 1
11:59:46 ERROR failed execution pid=46827 exit_code=1 error="installer: exit status 1"