Exception: databricks CLI is not installed

[noklam]

Context

I create a fresh environment today and start with a new kedro project and run into this issue

1. kedro new
2. pip install
3. pip install kedro-databricks
4. kedro init

Please select your cloud provider:

1. Azure
2. AWS
3. GCP ────────────────────────────────╮ │ /workspace/kedro/.venv/bin/kedro:8 in │ │ │ │ /workspace/kedro/kedro/framework/cli/cli.py:263 in main │ │ │ │ 260 │ cli_collection = KedroCLI( │ │ 261 │ │ project_path=_find_kedro_project(Path.cwd()) or Path.cwd() │ │ 262 │ ) │ │ ❱ 263 │ cli_collection() │ │ 264 │ │ │ │ /workspace/kedro/.venv/lib/python3.11/site-packages/click/core.py:1157 in call │ │ │ │ /workspace/kedro/kedro/framework/cli/cli.py:163 in main │ │ │ │ 160 │ │ ) │ │ 161 │ │ │ │ 162 │ │ try: │ │ ❱ 163 │ │ │ super().main( │ │ 164 │ │ │ │ args=args, │ │ 165 │ │ │ │ prog_name=prog_name, │ │ 166 │ │ │ │ complete_var=complete_var, │ │ │ │ /workspace/kedro/.venv/lib/python3.11/site-packages/click/core.py:1078 in main │ │ │ │ /workspace/kedro/.venv/lib/python3.11/site-packages/click/core.py:1688 in invoke │ │ │ │ /workspace/kedro/.venv/lib/python3.11/site-packages/click/core.py:1688 in invoke │ │ │ │ /workspace/kedro/.venv/lib/python3.11/site-packages/click/core.py:1434 in invoke │ │ │ │ /workspace/kedro/.venv/lib/python3.11/site-packages/click/core.py:783 in invoke │ │ │ │ /workspace/kedro/.venv/lib/python3.11/site-packages/click/decorators.py:45 in new_func │ │ │ │ /workspace/kedro/.venv/lib/python3.11/site-packages/kedro_databricks/plugin.py:114 in init │ │ │ │ 111 │ provider: str, │ │ 112 ): │ │ 113 │ """Initialize Databricks Asset Bundle configuration""" │ │ ❱ 114 │ write_bundle_template(metadata) │ │ 115 │ write_override_template(metadata, default, _PROVIDER_MAP.get(provider)) │ │ 116 │ if require_databricks_run_script(): │ │ 117 │ │ log = logging.getLogger(metadata.package_name) │ │ │ │ /workspace/kedro/.venv/lib/python3.11/site-packages/kedro_databricks/init.py:163 in │ │ write_bundle_template │ │ │ │ 160 │ project_path = metadata.project_path │ │ 161 │ log = logging.getLogger(package_name) │ │ 162 │ if shutil.which("databricks") is None: # pragma: no cover │ │ ❱ 163 │ │ raise Exception("databricks CLI is not installed") │ │ 164 │ │ │ 165 │ config = { │ │ 166 │ │ "project_name": package_name, │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ Exception: databricks CLI is not installed

[JenspederM]

Do you think we should do more in terms of documenting this, or is the new error message good enough?

What I really wanted to, was to add it as a system dependency. I'm just not entirely sure how to do that in Python...

[noklam]

I don't think it is possible to include system dependencies in a package. The alternative is using python sdk. I have not used it before so I do not know if it can replace the CLI

[JenspederM]

The python SDK doesn't support asset bundles (yet). See discussion in #8. If the SDK starts supporting asset bundles I'll refactor to remove any subprocess calls.