search

microsoft / azdo-databricks

A set of Build and Release tasks for Building, Deploying and Testing Databricks notebooks
MIT License
25 stars 30 forks source link

Executing other Databricks CLI commands #21

Closed charlespickettau closed 4 years ago

charlespickettau commented 4 years ago

These tools are great, but I'd like to be able to use other CLI commands as well. One example is to be able to deploy new Jobs from source control.

At the moment, I can deploy the Notebooks but when I try to run other commands using CLI, it tells me I haven't configured databricks yet.

Questions: 1) Is there an existing module/DevOps task to execute generic databricks CLI commands? 2) Is there a way to execute them using e.g. standard CLI

tkasu commented 4 years ago

Hey! I got this working by adding --profile AZDO to the cli commands, e.g.:

# Check that databricks cli is working
- script: databricks fs ls --profile AZDO
charlespickettau commented 4 years ago

Thanks @tkasu this works great.

NB I used the command databricks jobs list --profile AZDO as an alternative test of the Jobs endpoint