Running export-run as shell code in a Databricks notebook results in "You haven't configured the CLI yet!" error

[badjouras]

In a Databricks notebook, I ran

%pip install mlflow-export-import

%sh
export-run \
--run-id [a-run-id]\
--output-dir FileStore/shared_uploads/my.user@company.com

Got back

MLflow Version: 2.2.2 MLflow Tracking URI: databricks Options: run_id: d48964451a0940d7b2fb2fa96272305e output_dir: FileStore/shared_uploads/ricardo.cardoso@anova.com notebook_formats: WARNING: You haven't configured the CLI yet! Please configure by entering /local_disk0/.ephemeral_nfs/envs/pythonEnv-41f9030a-e1b2-4b46-b9ce-bcce6f3a95f3/bin/export-run configure Traceback (most recent call last): File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-41f9030a-e1b2-4b46-b9ce-bcce6f3a95f3/bin/export-run", line 8, in sys.exit(main()) File "/databricks/python/lib/python3.9/site-packages/click/core.py", line 1128, in call return self.main(args, kwargs) File "/databricks/python/lib/python3.9/site-packages/click/core.py", line 1053, in main rv = self.invoke(ctx) File "/databricks/python/lib/python3.9/site-packages/click/core.py", line 1395, in invoke return ctx.invoke(self.callback, ctx.params) File "/databricks/python/lib/python3.9/site-packages/click/core.py", line 754, in invoke return __callback(args, **kwargs) File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-41f9030a-e1b2-4b46-b9ce-bcce6f3a95f3/lib/python3.9/site-packages/mlflow_export_import/run/export_run.py", line 120, in main exporter = RunExporter( File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-41f9030a-e1b2-4b46-b9ce-bcce6f3a95f3/lib/python3.9/site-packages/mlflow_export_import/run/export_run.py", line 35, in init self.dbx_client = DatabricksHttpClient() File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-41f9030a-e1b2-4b46-b9ce-bcce6f3a95f3/lib/python3.9/site-packages/mlflow_export_import/common/http_client.py", line 72, in init super().init("api/2.0", host, token) File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-41f9030a-e1b2-4b46-b9ce-bcce6f3a95f3/lib/python3.9/site-packages/mlflow_export_import/common/http_client.py", line 22, in init raise MlflowExportImportException("MLflow host or token is not configured correctly") mlflow_export_import.common.MlflowExportImportException: MLflow host or token is not configured correctly

Running /local_disk0/.ephemeral_nfs/envs/pythonEnv-41f9030a-e1b2-4b46-b9ce-bcce6f3a95f3/bin/export-run configure, as suggested, results in

Usage: export-run [OPTIONS]
Try 'export-run --help' for help.

Error: Missing option '--run-id'.
MLflow Version: 2.2.2
MLflow Tracking URI: databricks

Do I need to to configure a .databrickscfg file to make the shell commands work inside a notebook?

[amesar]

Yes, you will need a ~/. databrickscfg to run the scripts from a shell.

[badjouras]

Even inside a Databricks notebook? If that's the case, I suggest documenting that in the README for Databricks Notebooks.

[amesar]

It is a general problem if you run Python scripts (or databricks CLI or mlflow CLI ) from inside a notebook. Nothing to do per se with mlflow-export-import. Best to raise the issue at https://community.databricks.com/

[amesar]

See https://github.com/mlflow/mlflow-export-import/blob/master/databricks_notebooks/scripts/Console_Scripts.py. Specify your PAT (token) via two secrets widgets which will in turn create the ~/.databrickscfg file