search

microsoft / azure-pipelines-tasks

Tasks for Azure Pipelines
https://aka.ms/tfbuild
MIT License
3.49k stars 2.61k forks source link

AzureFunctionApp@1 - unable to use zipDeploy with Linux consumption plan #13885

Closed tjrobinson closed 3 years ago

tjrobinson commented 3 years ago

Required Information

Entering this information will route you directly to the right team and expedite traction.

Question, Bug, or Feature?
Type: Bug

Enter Task Name: AzureFunctionApp@1

Environment

Issue Description

I am using the following YAML build definition:

# https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema

trigger: none # will disable CI builds entirely
pr: none # disable PR builds

variables:

  system.debug: true 

  # Azure Resource Manager connection created during pipeline creation
  azureSubscription: '6a18da4c-e905-4e1e-a0ea-45290698c1ac'

  # Function app name
  functionAppName: 'empactisazurefunctionsprintinguat-uks'

  # Environment name
  environmentName: 'empactisazurefunctionsprintinguat-uks'

  # Agent VM image name
  vmImageName: 'ubuntu-16.04'

stages:
- stage: Build
  displayName: Build stage
  jobs:  
  - job: Build
    displayName: Build
    pool:
      vmImage: $(vmImageName)

    steps:

    - bash: |
        if [ -f extensions.csproj ]
        then
            dotnet build extensions.csproj --output ./bin
        fi
      workingDirectory: '$(System.DefaultWorkingDirectory)/src/Empactis.AzureFunctions.Printing'

    - task: ArchiveFiles@2
      displayName: "Archive files"
      inputs:
        rootFolderOrFile: "$(System.DefaultWorkingDirectory)/src/Empactis.AzureFunctions.Printing"
        includeRootFolder: false
        archiveFile: "$(System.DefaultWorkingDirectory)/build$(Build.BuildId).zip"

    # https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/publish-build-artifacts?view=azure-devops
    - task: PublishBuildArtifacts@1
      inputs:
        pathToPublish: '$(System.DefaultWorkingDirectory)/build$(Build.BuildId).zip'
        artifactName: 'drop'

    # NOTE: This does not support removing settings, only adding or updating    
    - task: AzureAppServiceSettings@1
      displayName: Azure App Service Settings
      inputs:
        azureSubscription: '$(azureSubscription)'
        appName: $(functionAppName)
        appSettings: |
          [
            {
              "name": "ENABLE_ORYX_BUILD",
              "value": "true",
              "slotSetting": false
            },
            {
              "name": "FUNCTIONS_EXTENSION_VERSION",
              "value": "~3",
              "slotSetting": false
            },
            {
              "name": "FUNCTIONS_WORKER_RUNTIME",
              "value": "node",
              "slotSetting": false
            },
            {
              "name": "PLAYWRIGHT_BROWSERS_PATH",
              "value": "0",
              "slotSetting": false
            },
            {
              "name": "SCM_DO_BUILD_DURING_DEPLOYMENT",
              "value": "true",
              "slotSetting": false
            }
          ]

    # https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-function-app
    - task: AzureFunctionApp@1
      displayName: 'Azure Functions App Deploy'
      inputs:
        azureSubscription: '$(azureSubscription)'
        appType: functionAppLinux
        appName: $(functionAppName)
        deploymentMethod: zipDeploy
        package: '$(System.DefaultWorkingDirectory)/build$(Build.BuildId).zip'

This runs without errors, but the AzureFunctionApp step doesn't deploy the package using deploymentMethod: zipDeploy when the target is a consumption Function App. The relevant section of the logs being:

Got service connection details for Azure App Service:'empactisazurefunctionsprintinguat-uks'
##[debug][POST]https://login.windows.net/***/oauth2/token/
##[debug][GET]https://management.azure.com/subscriptions/801f8ac0-b532-4960-a473-77d2203857b6/resources?$filter=resourceType EQ 'Microsoft.Web%2FSites' AND name EQ 'empactisazurefunctionsprintinguat-uks'&api-version=2016-07-01
##[debug]Correlation ID from ARM api call response : 58a7cfef-4b17-40f1-9410-9a9d525038dc
##[debug]Resource Group: EmpactisUatLinux
##[debug][GET]https://management.azure.com/subscriptions/801f8ac0-b532-4960-a473-77d2203857b6/resourceGroups/EmpactisUatLinux/providers/Microsoft.Web/sites/empactisazurefunctionsprintinguat-uks/?api-version=2016-08-01
##[debug]Correlation ID from ARM api call response : 5eebcef8-7682-453a-b511-f70bd1e13f6f
##[debug]Sku: Dynamic
##[debug]Processed: ##vso[telemetry.publish area=TaskEndpointId;feature=AzureRmWebAppDeployment]{"endpointId":"6a18da4c-e905-4e1e-a0ea-45290698c1ac"}
##[debug]package=/home/vsts/work/1/s/build31921.zip
##[debug]Finding files matching input: /home/vsts/work/1/s/build31921.zip
##[debug]intially web config parameters :null
##[debug]This is zip package 
##[debug]runtimeStack=null
##[debug]deploymentMethod=zipDeploy
##[debug]Depolyment started for linux app service
##[debug]release.releaseId=undefined
##[debug]Artifact alias of package is: null
##[debug]set Azure_App_Service_Deploy_PackageArtifactAlias=
##[debug]Processed: ##vso[task.setvariable variable=Azure_App_Service_Deploy_PackageArtifactAlias;issecret=false;]
##[debug]Predeployment Step Started
##[debug]Deployment Step Started
##[debug][POST]https://management.azure.com/subscriptions/801f8ac0-b532-4960-a473-77d2203857b6/resourceGroups/EmpactisUatLinux/providers/Microsoft.Web/sites/empactisazurefunctionsprintinguat-uks/config/appsettings/list?api-version=2016-08-01
##[debug]Correlation ID from ARM api call response : fe19da54-d828-4a99-aac9-07734e2d1c66
##[debug]Storage Account is: empazurefunctionsuat
##[debug]set SAS_TOKEN=********
##[debug]Processed: ##vso[task.setvariable variable=SAS_TOKEN;issecret=true;]***
##[debug]SAS URL is: https://empazurefunctionsuat.blob.core.windows.net/azure-pipelines-deploy/package_1605196634175.zip?***
Trying to update App Service Application settings. Data: {"WEBSITE_RUN_FROM_PACKAGE":"https://empazurefunctionsuat.blob.core.windows.net/azure-pipelines-deploy/package_1605196634175.zip?***"}
##[debug]Value of WEBSITE_RUN_FROM_PACKAGE has been changed to https://empazurefunctionsuat.blob.core.windows.net/azure-pipelines-deploy/package_1605196634175.zip?***
##[debug][PUT]https://management.azure.com/subscriptions/801f8ac0-b532-4960-a473-77d2203857b6/resourceGroups/EmpactisUatLinux/providers/Microsoft.Web/sites/empactisazurefunctionsprintinguat-uks/config/appsettings?api-version=2016-08-01
##[debug]Correlation ID from ARM api call response : 843ea61b-6843-4a08-861a-34a4020fcfa1
Updated App Service Application settings.
Updated WEBSITE_RUN_FROM_PACKAGE Application setting to https://empazurefunctionsuat.blob.core.windows.net/azure-pipelines-deploy/package_1605196634175.zip?***
Syncing triggers for function app

If I use the same YAML definition but point it at a Linux Azure Function App which is hosted by an App Service Plan (i.e. not a consumption plan), the results are different (and what I want to happen for the consumption plan target):

Got service connection details for Azure App Service:'empactisazurefunctionsprintinguat-uks2'
##[debug][POST]https://login.windows.net/***/oauth2/token/
##[debug][GET]https://management.azure.com/subscriptions/801f8ac0-b532-4960-a473-77d2203857b6/resources?$filter=resourceType EQ 'Microsoft.Web%2FSites' AND name EQ 'empactisazurefunctionsprintinguat-uks2'&api-version=2016-07-01
##[debug]Correlation ID from ARM api call response : 9ca712ab-6970-4ee0-b751-bf6fa39f2c74
##[debug]Resource Group: EmpactisUatLinux
##[debug][GET]https://management.azure.com/subscriptions/801f8ac0-b532-4960-a473-77d2203857b6/resourceGroups/EmpactisUatLinux/providers/Microsoft.Web/sites/empactisazurefunctionsprintinguat-uks2/?api-version=2016-08-01
##[debug]Correlation ID from ARM api call response : 89274e48-5aee-4ecb-98d0-f914bf7cc663
##[debug]Sku: PremiumV2
##[debug]Processed: ##vso[telemetry.publish area=TaskEndpointId;feature=AzureRmWebAppDeployment]{"endpointId":"6a18da4c-e905-4e1e-a0ea-45290698c1ac"}
##[debug]package=/home/vsts/work/1/s/build31918.zip
##[debug]Finding files matching input: /home/vsts/work/1/s/build31918.zip
##[debug]intially web config parameters :null
##[debug]This is zip package 
##[debug]runtimeStack=null
##[debug]deploymentMethod=zipDeploy
##[debug]Depolyment started for linux app service
##[debug]release.releaseId=undefined
##[debug]Artifact alias of package is: null
##[debug]set Azure_App_Service_Deploy_PackageArtifactAlias=
##[debug]Processed: ##vso[task.setvariable variable=Azure_App_Service_Deploy_PackageArtifactAlias;issecret=false;]
##[debug]Predeployment Step Started
##[debug][POST]https://management.azure.com/subscriptions/801f8ac0-b532-4960-a473-77d2203857b6/resourceGroups/EmpactisUatLinux/providers/Microsoft.Web/sites/empactisazurefunctionsprintinguat-uks2/config/publishingcredentials/list?api-version=2016-08-01
##[debug]Correlation ID from ARM api call response : 702fe336-970d-4bf9-be2d-2543f0c63ebf
##[debug]set AZURE_APP_SERVICE_KUDU_production_PASSWORD=********
##[debug]Processed: ##vso[task.setvariable variable=AZURE_APP_SERVICE_KUDU_production_PASSWORD;issecret=true;]***
##[debug]Deployment Step Started
##[debug]Processed: ##vso[telemetry.publish area=TaskDeploymentMethod;feature=AzureWebAppDeployment]{"deploymentMethod":"Zip Deploy"}
##[debug]Performing Linux built-in package deployment
Trying to update App Service Application settings. Data: {"WEBSITES_ENABLE_APP_SERVICE_STORAGE":"true"}
##[debug][POST]https://management.azure.com/subscriptions/801f8ac0-b532-4960-a473-77d2203857b6/resourceGroups/EmpactisUatLinux/providers/Microsoft.Web/sites/empactisazurefunctionsprintinguat-uks2/config/appsettings/list?api-version=2016-08-01
##[debug]Correlation ID from ARM api call response : afa0d298-b65d-46f2-9842-6a291a64bd6a
##[debug]Value of WEBSITES_ENABLE_APP_SERVICE_STORAGE has been changed to true
##[debug][PUT]https://management.azure.com/subscriptions/801f8ac0-b532-4960-a473-77d2203857b6/resourceGroups/EmpactisUatLinux/providers/Microsoft.Web/sites/empactisazurefunctionsprintinguat-uks2/config/appsettings?api-version=2016-08-01
##[debug]Correlation ID from ARM api call response : 98f2fbbb-46ef-469a-8d7e-ac5ad23da488
Updated App Service Application settings and Kudu Application settings.
##[debug][GET]https://management.azure.com/subscriptions/801f8ac0-b532-4960-a473-77d2203857b6/resourceGroups/EmpactisUatLinux/providers/Microsoft.Web/sites/empactisazurefunctionsprintinguat-uks2/config/slotConfigNames?api-version=2016-08-01
##[debug]Correlation ID from ARM api call response : f7630987-8c77-40ed-ab9b-22c0c3dd898d
Package deployment using ZIP Deploy initiated.

Task logs

log.zip

tjrobinson commented 3 years ago

I've worked around this by using the Azure Functions Core Tools instead of the AzureFunctionApp task:

- task: FuncToolsInstaller@0
  displayName: 'Install Azure Functions Core Tools'

- task: AzureCLI@2
  displayName: 'func azure functionapp publish'
  inputs:
    azureSubscription: 'Empactis (801f8ac0-b532-4960-a473-77d2203857b6)'
    scriptType: bash
    scriptLocation: inlineScript
    inlineScript: |
     func azure functionapp publish empactisazurefunctionsprintinguat-uks --build remote
    workingDirectory: src/Empactis.AzureFunctions.Printing
    failOnStandardError: true
AmrutaKawade commented 3 years ago

Currently only Run from package deploy using url is supported for Linux Function apps running on consumption plan.

https://docs.microsoft.com/en-us/azure/azure-functions/functions-deployment-technologies#external-package-url

External package URL is the only supported deployment method for Azure Functions running on Linux in the Consumption plan, if the user doesn't want a remote build to occur.

remote builds are not supported currently in task.

As Functions Core Tool by default enables remote builds its working for you

@N-Usha can you look into this enhancement request?

m1nkeh commented 3 years ago

hmm, i'm not sure if i have exactly the same issue or just a tightly related one but this line in the debug bothers be:

Value of WEBSITE_RUN_FROM_PACKAGE has been changed to https://empazurefunctionsuat.blob.core.windows.net/azure-pipelines-deploy/package_1605196634175.zip?

i am deploying Python to a Linux FunctionApp and the constant updating of this setting each time with a new *.zip file that has been published to blob storage is really winding me up - mainly as it's not part of my infrastructure deployment (terraform)

when working in VSCode, with the func azure functionapp publish command, this does not occur..

At this point, the only option i can see is to exclude this config setting from terraform, as the func azure functionapp publish command is a combination of build and publish which i like even less 😐

Additionally, the -nozip commands seems to do precisely nothing, and overwrites the file in the scm-releases container on blob storage with a new zip anyway?? maybe that's because i am on a Linux FunctionApp?

Why is this functionality so different please?

m1nkeh commented 3 years ago

after waaaaaaaaaaaay too much time doing trial and error deployments, i eventually figured out this will work:

- task: AzureRmWebAppDeployment@4
  displayName: Deploy to FunctionApp
  inputs:
    ConnectionType: "AzureRM"
    azureSubscription: "Your-Linked-Service-Name"
    appType: "webAppLinux"
    WebAppName: "Your-Function-App-Name"
    package: "$(System.ArtifactsDirectory)/**/*.zip"

This will allow me to deploy a Python project to a FunctionApp running on consumption plan, it uses SCM just like the normal deployment 🎉

there a has to be a better way at *.yaml discovery.. it a total mess, and now Azure DevOps has removed the button that used to allow you to export YAML!

image

jusbuc2k commented 3 years ago

I have the same problem. Using the AzureRmWebAppDeployment task above doesn't work for me for a .NET Core 3.1 function app. I think it fails because it's trying to do a build on deploy, and the SDK doesn't exist on the function app node?

It seems to me that there needs to be a way to persist the URL in the WEBSITE_RUN_FROM_PACKAGE app setting across ARM deployments, so that the existing package stays in place.

j2020v commented 3 years ago

Also having the same issue. I am deploying a consumption based linux function app and it fails to perform a swap slot because it is expecting these in the appsettings

[error]Error: Failed to swap App Service 'func-app' slots - 'func-app-slot1' and 'production'. Error: BadRequest - Storage access failed. WEBSITE_CONTENTAZUREFILECONNECTIONSTRING or WEBSITE_CONTENTSHARE appsetting is missing (CODE: 400)

According to the documentation: "Only used when deploying to a Premium plan or to a Consumption plan running on Windows. Not supported for Consumptions plans running Linux." https://docs.microsoft.com/en-us/azure/azure-functions/functions-app-settings#website_contentazurefileconnectionstring

So why does it expect these in the appsettings when I've specified the service type I am deploying is functionAppLinux?

This is currently what is in my YML file:

j2020v commented 3 years ago

cc @mhoeger

m1nkeh commented 3 years ago

I have the same problem. Using the AzureRmWebAppDeployment task above doesn't work for me for a .NET Core 3.1 function app. I think it fails because it's trying to do a build on deploy, and the SDK doesn't exist on the function app node?

It seems to me that there needs to be a way to persist the URL in the WEBSITE_RUN_FROM_PACKAGE app setting across ARM deployments, so that the existing package stays in place.

What language? C# i assume? I will test..

jusbuc2k commented 3 years ago

@m1nkeh Yes, c#, aspnet core 3.1 function app.

github-actions[bot] commented 3 years ago

This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days

Rutix commented 3 years ago

This is really annoying since our ARM templates screw up the function in consumption everytime (since they remove all application settings which are not specified). Any workaround for this?

mikemutpo commented 1 year ago

@Rutix I've found that publishing with Azure Function Core Tools CLI is 100% reliable compared to this nonsense.

tomaustin700 commented 1 year ago

I don't understand how this it's still not possible to publish to a Linux consumption plan function using zipDeploy. We've wasted hours on this until finally finding this issue. The task should at least provide a meaningful error pointing at this being the issue. What needs to be done to get this issue reopened?

sylnsr commented 1 year ago

This is really annoying since our ARM templates screw up the function in consumption everytime (since they remove all application settings which are not specified). Any workaround for this?

Not within AzDOS, no. Best work around we found was having AzDOS call Azure CLI: https://learn.microsoft.com/en-us/azure/azure-functions/deployment-zip-push#cli .. this was the only way to get it working almost flawlessly.

AndreaCuneo commented 1 year ago

Same here. Spent couple of days do debug this. And for me this is critical as create bug in production.

The issue from my perspective stems from the fact that we employ IaC and we deploy the Infrastructure via a separate release from the Code applications. Application Settings, mainly connection strings, are managed by IaC. When ARM template are deployed the RESET the AppSettings since the Microsoft.Web/site/config resource replace all the settings. So after each Deploy of the Infrastructure, our applications disappear.

Well, since RUN_FROM_PACKAGE=1 is not supported on Linux Consumption, let's use old reliable ZipDeploy, supposed to be identical to the func publish the VSCode (the IDE) execute.

Configuring in the YAML deploymentMethod: zipDeploy silently choose to ignore it and continue with RUN_FROM_PACKAGE=<URL>. Not a WARN, not an ERROR, not event an INFO log ...

Then, understanding that this is clearly a Microsoft Bug, I found this thread.

Thanks @tjrobinson @m1nkeh for providing a solution