search

databrickslabs / cicd-templates

Manage your Databricks deployments and CI with code.
Other
201 stars 101 forks source link

dbx execute doesn't work on ml cluster #51

Closed chinwobble closed 3 years ago

chinwobble commented 3 years ago

If you attempt to run dbx execute it won't work on ml clusters.

Since 

%pip install -U -r {path}

Is only supported on databricks runtime and not databricks ML Cluster

renardeinside commented 3 years ago

Hi @chinwobble !

I'm not sure if I understand your issue properly. %pip is supported as per this documentation. In the quickstart example, we use 7.3 ML LTS. Could you please specify the version where it is not working?

chinwobble commented 3 years ago

I tested it on a ML Cluster 6.4 LTS and it wasn't working. My mistake, happy to close this issue.