DataThirstLtd / databricks.vsts.tools

VSTS Deployment Tasks for Databricks Objects
MIT License
18 stars 3 forks source link

Tasks only work in a Windows agent, even though the powershell modules are cross-platform #52

Open badjouras opened 2 years ago

badjouras commented 2 years ago

The JSON for each task restricts it to Windows agents: image https://github.com/DataThirstLtd/databricks.vsts.tools/blob/master/deployScriptsTask/task.prod.json

Running them on a Ubuntu agent yields an error message:

##[error]The current operating system is not capable of running this task. That typically means the task was written for Windows only. For example, written for Windows Desktop PowerShell.

However, the powershell modules from azure.databricks.cicd.tools support Powershell Core and, hence, work on Linux agents.

The JSON files should be updated to remove the restriction to Windows agents.

jessehouwing commented 2 years ago

That won't help. Azure Pipelines can't execute a PowerShell task on PowerShell Core and will always use Windows PowerShell available on the host. For these tasks to work on a non-windows agent, thy need to be rewritten in Node using the Azure Pipelines Task Lib.

badjouras commented 2 years ago

Are you referring to a pipeline task defined in a piepline YAML, or to a marketplace task?

I am using pipeline PowerShell tasks on a ubuntu agent with no problem:

image image

The databricksDeployDBFSFiles.ps1 is simply a modified, simpler version of https://github.com/DataThirstLtd/databricks.vsts.tools/blob/master/deployDBFSFilesTask/vstsDeployDBFSFile.ps1