search

microsoft / azdo-databricks

A set of Build and Release tasks for Building, Deploying and Testing Databricks notebooks
MIT License
25 stars 31 forks source link

Configure DataBricks Task Fails with no error log #48

Open mayankthebest opened 2 years ago

mayankthebest commented 2 years ago

image As shown in the above image, the configure Databricks CLI task fails silently leading to cascading failures.

kurowknr commented 2 years ago

It seems that the new version of Databrlicks Cli somehow corrupts the task. My workaround: manually install pip install databricks-cli before invoking the task

akshayrohilla commented 2 years ago

It seems to be working with the MS Hosted Ubuntu Agents.

MS-Trev commented 2 years ago

Note that the DevLabs task page recommends Ubuntu for this: “It's strongly recommend that you use Hosted Ubuntu 1604 for your pipelines.”

The DB CLI requires Python 3.6 or higher, per https://docs.databricks.com/dev-tools/cli/index.html

If I add the “Use Python version” task to my pipeline, tell it to use “Version spec” 3.6, and run that task before the DevLabs “Configure Databricks” task (another recommendation of the VS Market page for this extension), it works again on the MS-Hosted WIN images 2016 & 2019.

The "windows-2022" image only has Python 3.7.9 and other higher versions on it so same approach fails there, but I'd expect it to work too if the DB CLI doc is correct ("Python 3.6 or higher). No sure what's up there.

cvonredapt commented 2 years ago

Thank you, @MS-Trev. Setting Python to 3.6.x before the Databricks Config task solved the problem.

Looks like we should work on moving our pipelines to Ubuntu, as well.

MJ029 commented 2 years ago

Thank you, @MS-Trev. Setting Python to 3.6.x before the Databricks Config task solved the problem.

Looks like we should work on moving our pipelines to Ubuntu, as well.

This Works for me, Thanks @cvonredapt.

ulriksenstian commented 2 years ago

I think both this and #49 are related

Rajendramca commented 1 year ago

Currently it is failing in deploy notebooks to work space 2023-05-05T10:20:24.4970460Z [command]C:\hostedtoolcache\windows\Python\3.9.13\x64\Scripts\databricks.exe workspace import_dir -o --profile AZDO D:\a\r1\a_Dev_ADB_Build\idm_adb /Shared 2023-05-05T10:20:25.3714253Z Error: AttributeError: type object 'Retry' has no attribute 'DEFAULT_METHOD_WHITELIST'

cvonredapt commented 1 year ago

@Rajendramca, had the same experience. Set the python version to 3.6 on your client