Azure / azure-functions-powershell-worker

PowerShell language worker for Azure Functions.
MIT License
203 stars 52 forks source link

azure cli from azure function powershell #221

Open dariuszbz opened 5 years ago

dariuszbz commented 5 years ago

How to run azure cli from azure function powershell?

daxian-dbw commented 5 years ago

Can you use the Azure PowerShell module instead?

dariuszbz commented 5 years ago

Not from functions. When I'm loading azurerm module for SQL scaling (thats my task) and than login/connected nf to azure I'm getting error message.

On Tue, 14 May 2019, 06:50 Dongbo Wang, notifications@github.com wrote:

Can you use the Azure PowerShell module instead?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Azure/azure-functions-powershell-worker/issues/221?email_source=notifications&email_token=AEVINAV63GQNJIQXPMGA5FDPVJHKLA5CNFSM4HMQGFKKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODVKKXCI#issuecomment-492088201, or mute the thread https://github.com/notifications/unsubscribe-auth/AEVINAUKDXLQHAWLMR72KDTPVJHKLANCNFSM4HMQGFKA .

dariuszbz commented 5 years ago

Btw. it works (4ex: Get-AzContext ) when running from vscode, but not after deployment into azure function.

On Tue, 14 May 2019 at 06:50, Dongbo Wang notifications@github.com wrote:

Can you use the Azure PowerShell module instead?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Azure/azure-functions-powershell-worker/issues/221?email_source=notifications&email_token=AEVINAV63GQNJIQXPMGA5FDPVJHKLA5CNFSM4HMQGFKKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODVKKXCI#issuecomment-492088201, or mute the thread https://github.com/notifications/unsubscribe-auth/AEVINAUKDXLQHAWLMR72KDTPVJHKLANCNFSM4HMQGFKA .

dariuszbz commented 5 years ago

Ok I found: first on your VM/workstation: Connect-AzAccount Save-AzContext

copy context json file to your azure function file system.

in your functions: Import-AzContext -Path "D:\home\site\wwwroot\<you function name>\azctx.json" $az = Get-AzContext write-host $az

works.

On Tue, 14 May 2019 at 11:19, Dariusz Bzówka daribzow@gmail.com wrote:

Btw. it works (4ex: Get-AzContext ) when running from vscode, but not after deployment into azure function.

On Tue, 14 May 2019 at 06:50, Dongbo Wang notifications@github.com wrote:

Can you use the Azure PowerShell module instead?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Azure/azure-functions-powershell-worker/issues/221?email_source=notifications&email_token=AEVINAV63GQNJIQXPMGA5FDPVJHKLA5CNFSM4HMQGFKKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODVKKXCI#issuecomment-492088201, or mute the thread https://github.com/notifications/unsubscribe-auth/AEVINAUKDXLQHAWLMR72KDTPVJHKLANCNFSM4HMQGFKA .

daxian-dbw commented 5 years ago

Did you enable MSI for your FunctionApp? Once you enable MSI and assign the necessary roles to your identity object, the default profile.ps1 will log you in Azure automatically.

dariuszbz commented 5 years ago

It didn't work.

I can try any step by step guide if you send me step by step instructions.

The only way it works for me is the one I described.

On Tue, 14 May 2019, 18:05 Dongbo Wang, notifications@github.com wrote:

Did you enable MSI for your FunctionApp? Once you enable MSI and assign the necessary roles to your identity object, the default profile.ps1 will log you in Azure automatically.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Azure/azure-functions-powershell-worker/issues/221?email_source=notifications&email_token=AEVINASTU533Q37LCGD5T33PVLWODA5CNFSM4HMQGFKKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODVMEMYQ#issuecomment-492324450, or mute the thread https://github.com/notifications/unsubscribe-auth/AEVINATGTJGGPQQODMRQYPDPVLWODANCNFSM4HMQGFKA .

anirudhgarg commented 5 years ago

az cli is not available currently out the box in the functions powershell environment. This is a feature ask and we are looking into it.

AnatoliB commented 4 years ago

In the meantime, the following workaround has been verified:

After that, the following function runs successfully:

param($Request, $TriggerMetadata)
az login --identity
$FunctionApps = az functionapp list | ConvertFrom-Json
$FunctionApps.Name
VictorioBerra commented 4 years ago

@AnatoliB can you elaborate on where to to upload the CLI2 folder?

You said:

to the file share for the function app using storage explorer

I am using SE and I do not see a file share.

AnatoliB commented 4 years ago

@VictorioBerra When you create a PowerShell Function app, a storage account is normally created in the same resource group, and this storage account contains an automatically created file share. You can access it in multiple ways, for example:

  1. Using Storage Explorer: In the Portal, find the Function app and click on the Overview tab, then follow the link to the resource group. You will find a storage account in this resource group. Open this storage account in Storage Explorer, go to file shares and find a share with an auto-generated name. I recommend uploading CLI2 to the data directory.
  2. Using Kudu: In the Portal, find the Function app and click Platform features -> Advanced tools (Kudu), then select Debug console -> PowerShell, and then you can drag-and-drop zip files to be uploaded there.
VictorioBerra commented 4 years ago

@AnatoliB I have no file shares in mine. Does it matter that i'm hosting an existing app service plan on Windows? As for Kudu, if I upload the CLI2 as a zip I assume I simply unzip using a PS command?

AnatoliB commented 4 years ago

@VictorioBerra I see. Yes we use a different kind of file storage on app service plan, so it is not on Azure Files, you are right. In this case, the Kudu method should still work. When you drag-and-drop a zip file into the browser window, Kudu will automatically unzip the content into the current location, preserving the directory structure in the zip.

VictorioBerra commented 4 years ago

@AnatoliB I must have done something wrong with Kudu, I zipped the CLI2 directory, dragged the zip into my browser, it uploaded successfully but never extracted. This was about 30 minutes ago now image

VictorioBerra commented 4 years ago

I was able to unzip manually in the PS console by just using unzip CLI2.zip. Its not done unzipping yet but it seems to be working.

EDIT: No luck. When I tried to do az login --identity in my run.ps1 I got the following error:

Exception: The term 'd:\home\data\CLI2\wbin\az.cmd' is not recognized as the name of a cmdlet, function, script file, or operable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.

I have Set-Alias -Name az -Value "d:\home\data\CLI2\wbin\az.cmd" in my profile.ps1

VictorioBerra commented 4 years ago

I gave up and used the REST API to accomplish my task. IMO the PS Functions are severely lacking without the AZ CLI. The Azure Powersehll Module is not enough. For example, what is the equivalent of this in APM? https://docs.microsoft.com/en-us/cli/azure/webapp/webjob/triggered?view=azure-cli-latest#az-webapp-webjob-triggered-list

@anirudhgarg what is the ETA of the feature ask? Do you have a link we can use to track the progress of the feature. Like on https://feedback.azure.com/?

dariuszbz commented 4 years ago

@VictorioBerra by now it looks like the only way. see: anirudhgarg commented on 28 May 2019 • az cli is not available currently out the box in the functions powershell environment. This is a feature ask and we are looking into it.

g0pinath commented 4 years ago

I am looking to use az cli for some features that are not available in PS yet. https://docs.microsoft.com/en-us/cli/azure/container?view=azure-cli-latest#az-container-start Please let me know if there is an alternate. Az PS commands works from within the functions.

bhushanI commented 4 years ago

@g0pinath Which Az PS commands are working for you within the functions? I am trying this command "az login --identity", still does not work for me. I have followed the steps given by @AnatoliB

bhushanI commented 4 years ago

@anirudhgarg could you please tell when this feature will be available?

prasanthchil commented 4 years ago

Hi Team, I followed the above steps of uploading azure cli from mac book onto azure powershell functions and updated the profile.ps1, but still receive az command in not found. It would help if a detailed step by step instructions are provided for the workaround till the time microsoft provide inbulilt az cli support in azure functions.Thanks.

abhilashredd commented 4 years ago

Its been more than a year and We still do not have this feature.

MasilamaniSeenivasan commented 4 years ago

Hi All,

We able to run the Azure CLI commands from Function App (Powershell) But we are facing PERFORMANCE issue, like "az account set" command take minimum 2 mins and "az aks show" command takes minimum 3 mins. we achieved our requirement by using these commands inside the loop. so it impacts much.

  1. Why the Azure CLI commands taking long time in Azure Function App? because in Windows Azure CLI the "az account set" command takes around 3 secs and "az aks show" command takes around 6 secs to run.
  2. Do we have any option to improve the performance of the Azure CLI commands from Azure Function App?

Please let me know if anything is not clear and share your thoughts on this.

Thanks, Seenivasan.M

AnatoliB commented 4 years ago

@MasilamaniSeenivasan 2-3 min sounds way more than expected. How do you deploy Azure CLI? Do you include it in your app content, or upload it separately? What is the target location? Also, how do you invoke it and how you measure the time? Could you please share the code?

MasilamaniSeenivasan commented 4 years ago

used in loop to achieve our functionality

(Get-Date).ToUniversalTime()

set subscription of current AKS

az account set -s (Get-Date).ToUniversalTime()

get AKS's msi objectId

$objId = $(az aks show -g -n --query identityProfile.kubeletidentity.objectId -o tsv) (Get-Date).ToUniversalTime()

Please let me know if anything else required.

OwainJ commented 3 years ago

Is this still not supported?

I have Az listed in my requirements.psd1 image

And I also invoked Enable-AzureRmAlias in my profile.ps1 image

But I'm still getting the "The term 'az' is not recognized" error... I don't particularly want to go down the route of having to manually upload the CLI2 folder, as that just seems gross.

AnatoliB commented 3 years ago

@OwainJ Please note that Azure PowerShell (Az/AzureRm modules) and Azure CLI are separate and independent tools. The requirements.psd1 and profile.ps1 code in your comment refers specifically to Azure PowerShell. In order to use Azure CLI, the workarounds mentioned above are still required.

Please do let us know, however, which capabilities of Azure CLI you are interested in. Can this be currently achieved by Azure PowerShell instead? Chances are, extending Azure PowerShell is a better idea, but this depends on the details. Your feedback will help us prioritize, thank you.

OwainJ commented 3 years ago

Hi @AnatoliB, thanks for your reply!

Ahh, I see, thanks for clearing that up! I'm very much a newbie with this stuff, so apologies in advance if anything I say is completely wrong.

Basically, I've been given a PowerShell command that creates a container instance and pushes the latest release image of a crawler to it. That command currently runs as an inline script (Azure CLI task) on an Azure DevOps release pipeline, which is triggered on a timer schedule.

This is the script that we have in the release pipeline Azure CLI task: az container create --resource-group $(resourceGroupName ) --name crawler-incremental-$(deploymentName) --image $(acrLoginServer)/crawlers/incremental:$(Build.BuildId) --dns-name-label crawler-incremental-$(deploymentName) --ports 80 --registry-login-server $(acrLoginServer) --registry-username $(acrName) --registry-password $(acrPassword) --cpu 4 --memory 16 --location $(location) --azure-file-volume-account-key $(storageAccountKey) --azure-file-volume-account-name $(storageAccountName) --azure-file-volume-mount-path /crawler/ --azure-file-volume-share-name crawler-incremental-$(deploymentName) -fs --restart-policy Never

I was hoping that I could move this to a Timer based PowerShell Function App, so that it can then keep it in source control, and deploy it to our client's azure resource group, rather than having it just float around in a DevOps release pipeline.

anirudhgarg commented 3 years ago

Yes that should be possible using the corresponding Azure PowerShell Module for Azure Container Instances.

Have a look here: https://docs.microsoft.com/en-us/azure/container-instances/container-instances-quickstart-powershell https://docs.microsoft.com/en-us/azure/container-instances/container-instances-quickstart-powershell#create-a-container

OwainJ commented 3 years ago

@anirudhgarg Thank you so much, that worked perfectly!

Nilsas commented 3 years ago

Still looking for this feature, I'm looking to run an ad-hoc command in Azure Container Instance using az container exec from Azure CLI as Powershell Az module lacks this functionality https://docs.microsoft.com/en-us/powershell/module/az.containerinstance/?view=azps-5.0.0#container-instances

DanielSSilva commented 3 years ago

I'm in a similar position as @Nilsas. I would like to have a function app that has control over containers instances and be able to start/stop them. Is this being worked on?

sebader commented 3 years ago

@Nilsas not sure if this is still relevant for you, but there is often times the workaround of using Invoke-AzRestMethod and the REST API of a service directly. For example to execute on ACI: https://docs.microsoft.com/en-us/rest/api/container-instances/containers/executecommand

msidave commented 3 years ago

Please do let us know, however, which capabilities of Azure CLI you are interested in.

@AnatoliB yaml file import, az container create --file options.yml, does not exist in Azure Powershell. Looking for the same thing as @DanielSSilva , the ability to create, start, stop, and remove container instances.

az cli supports secure environment variables and volume secret mounts through the yaml file import.

PingMudit commented 3 years ago

Hi @AnatoliB I want to create keyvault secret refences to app configuration. I don't see a powershell cmdlet available which can be used for this. I wanted to use event grid function so that when a new secret gets created, the function creates it's reference as well in app configuration. References can be set using CLI but it's not yet supported in functions. Will the above workaround of uploading CLI2.zip to data directory work for this case too?

AnatoliB commented 3 years ago

@PingMudit If you are looking for a way to update app settings in general, Update-AzFunctionAppSetting is probably what you need. Uploading CLI to the file system works in general, though I cannot guarantee it will satisfy all possible scenarios.

PingMudit commented 3 years ago

@AnatoliB Unfortunately, with Update-AzFunctionAppSetting, only app configuration modifications are supported.. I wanted to create reference of keyvault secrets under App Configuration -> Configuration Explorer which isn't supported by above cmdlet. I was wondering if the workaround of uploading CLI2.zip will help here in using CLI within Azure Powershell functions?

AnatoliB commented 3 years ago

@PingMudit Ok, I see what you mean now. I don't know if this is currently covered by any Azure PS cmdlets. However, if it is covered by CLI, try to upload it as described above. Like I said before, this workaround allows using CLI in PowerShell Functions in general. I don't know if anyone tested adding keyvault secret references specifically, but there is a good chance it will just work.

PingMudit commented 3 years ago

@AnatoliB Thank you for your quick response! I tried to set alias in profile.ps1 but it kept giving me error that az is not recognized. Then I added Set-Alis directly in function. SetAlias

This seems to be working as I can see output for command az login --identity. However when I try to set reference of secret using command az appconfig kv set-keyvault -n SecretReferences --key SQLPass --label labtest --y --secret-identifier "https://lab-mdkv21.vault.azure.net/secrets/SQLPass/a59eb6f74d944e6cb2c4e48b6aea5330" --y, it's giving me error as shown in attached pic. Could you please assist if there is something I can do to fix this? I am not sure what this error means. Error

AnatoliB commented 3 years ago

@PingMudit I would suggest following up with the Az CLI owners on this. The az command seems to be invoked correctly (from the PowerShell perspective), but az throws an exception (seems to be coming from the knack package).

PingMudit commented 3 years ago

@AnatoliB Thanks for your guidance on this. I changed my automation flow as there seems to be limitations with cmdlets for App Configuration. However when I use Get-AzAppConfigurationStore. Screenshot of error attached. function not supporting cmdlet Kindly let me know if I should add app configuration module in requirements.psd1

AnatoliB commented 3 years ago

@PingMudit The message on your screenshot explains the reason, and the link points to the instructions :-)

kavyabgowda commented 3 years ago

I tried manually uploading CLI2 inside folder as well as in data folder both worked. But when I zipped the site folder and publish the zip file using terraform it's not working..its justshowing information logs not even errors. Is there any way to publish the az modules completely along with code to function app using terraform

hobbesuk commented 3 years ago

I've also hit this limitation on not having access to the az cli commands, I've tried converting to using the Azure Powershell module but I can't find an equivalent way of running the following:

az monitor app-insights query

The powershell module doesn't have the ability to run queries, the recommended approach is to call the rest API of application insights, but I have no API key for my Application insights and do not really want to create one for each instance we have when the az cli appears to not need one.

AlexHimself commented 3 years ago

@OwainJ Please note that Azure PowerShell (Az/AzureRm modules) and Azure CLI are separate and independent tools. The requirements.psd1 and profile.ps1 code in your comment refers specifically to Azure PowerShell. In order to use Azure CLI, the workarounds mentioned above are still required.

Please do let us know, however, which capabilities of Azure CLI you are interested in. Can this be currently achieved by Azure PowerShell instead? Chances are, extending Azure PowerShell is a better idea, but this depends on the details. Your feedback will help us prioritize, thank you.

I'm trying to use Azure Functions (PowerShell) + App Configs, but I cannot enumerate the Key/Value pairs in the config store using the Azure PowerShell, so I can't use the config store unless I do the entire workaround...

kk921dbg commented 3 years ago

Any update on this enhancement as will be available to use in automation runbooks soon?

davidljuba15031979 commented 2 years ago

I need Azure CLI to download Azure Universal Package from Azure Function App in Powershell.

I need to execute: az artifacts universal download ... command to download artifact from Azure DevOps, due to limitation with Azure Powershell mentioned here:

https://github.com/MethodsAndPractices/vsteam/issues/151

It is also stated in the documentation that you have to use the Azure CLI currently:

You must use the Azure CLI to download the package. Azure DevOps doesn't support direct HTTP/HTTPS download links or other ways to download the package.

In local environment Azure Function (debugging in Ubuntu) is working correctly (I have installed locally Azure CLI), but it does not work in Azure Cloud (--os-type Windows).

CC: @AnatoliB

smholvoet commented 2 years ago

@AnatoliB @anirudhgarg Is Azure CLI support for PowerShell-based Azure Functions coming any time soon? 👀

Similar to @davidljuba15031979's request, we have a use case which heavily relies on the az devops module. This module doesn't have any good alternatives apart from reinventing the wheel using the Azure DevOps REST API or 3rd-party PowerShell modules which are wrappers for said REST API. This is a real shame considering how powerful the Azure CLI module is.

AnatoliB commented 2 years ago

cc @stefanushinardi @Francisco-Gamino

@smholvoet Have you tried to deploy Az CLI to the storage account associated with your Functions app (for example, using the instructions at https://github.com/Azure/azure-functions-powershell-worker/issues/221#issuecomment-546039332) or even include Az CLI binaries into your app content?

It is true that we don't currently provide Az CLI by default, but we also don't block it if you manage to deploy Az CLI to your app yourself. The extra steps are generally quite feasible, and they are not really different from deploying any other command-line tool. Making Az CLI deployed to PowerShell Functions by default is somewhat complicated and it's unlikely to happen soon, so I recommend all users to deploy Az CLI to their app instead

So, have you tried and something didn't work, or the instructions are not good enough, or there is something technically blocking this approach in your scenario?

If you get to the point when the az command is apparently invoked but reports an error (for example, like here: https://github.com/Azure/azure-functions-powershell-worker/issues/221#issuecomment-808140795), I would suggest following up with Az CLI owners by filing a specific issue at https://github.com/Azure/azure-cli/issues.

@stefanushinardi Let's consider documenting Az CLI deployment guidance properly.

smholvoet commented 2 years ago

@AnatoliB Appreciate the quick response and honesty 💪

I did manage to copy my CLI2 folder to the storage account using Azure Storage Explorer client. The Azure Storage Explorer in the portal, as well as uploading via Kudu both failed (some folders were not getting unzipped properly):

image

I then added an alias which points to my az.cmd. I then added the following snippet to my run1.ps1 so I could verify everything works as expected:

# verify Azure CLI works
az version
az extension add --name azure-devops
az extension list

az version seems to work fine:

image

However, installing / loading the azure-devops extension seems to fail, which then messes up the rest of the run as my script references multiple commands which use the azure-devops Azure CLI module. However, the output would suggest it did manage to load the azure-devops extension:

image

image

The exact same script which is being referenced in the Function app runs fine locally, as well as on the Microsoft-hosted Azure DevOps agents.

AnatoliB commented 2 years ago

@smholvoet Have you tried to use the --debug switch as the first error message suggests? In any case though, the error message is produced by Az CLI, so I think this is where the investigation should start. Depending on the results, we may be able to adjust the Function execution environment if Az CLI requires something special, but we need Az CLI diagnostics to understand the problem first. So, please report this to https://github.com/Azure/azure-cli/issues.