[BUG]: AzureFunctionApp@2 fails to deploy if Private EndPoints exists even with correct network permissions on scm

[flytzen]

Task name

AzureFunctionApp@2

Task version

2.221.105

Environment type (Please select at least one enviroment where you face this issue)

• [ ] Self-Hosted
• [X] Microsoft Hosted
• [ ] VMSS Pool
• [ ] Container

Azure DevOps Server type

dev.azure.com (formerly visualstudio.com)

Azure DevOps Server Version (if applicable)

No response

Operation system

ubuntu-latest

Task log

Starting: Deploy Management Subsystem to obxxx-as
==============================================================================
Task         : Azure Functions Deploy
Description  : Update a function app with .NET, Python, JavaScript, PowerShell, Java based web applications
Version      : 2.221.105
Author       : Microsoft Corporation
Help         : https://aka.ms/azurefunctiontroubleshooting
==============================================================================
Got service connection details for Azure App Service:'obxx-as'
(node:1611) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
##[error]Function app has private endpoint(s). But you are not running this pipeline from a self-hosted agent that has access to the Functions App. Relevant documentation: https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/agents?view=azure-devops&tabs=browser#install
NOTE: Function app is VNet integrated.
##[error]Error: Networking validation for the Function app and Storage account has failed. Please review all error messages.
Successfully updated deployment History at https://obxxx-as.scm.azurewebsites.net/api/deployments/331686839307234
App Service Application URL: https://obxx-as.azurewebsites.net
Finishing: Deploy Management Subsystem to obxx-as

Relevant log output

##[error]Function app has private endpoint(s). But you are not running this pipeline from a self-hosted agent that has access to the Functions App. Relevant documentation: https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/agents?view=azure-devops&tabs=browser#install
NOTE: Function app is VNet integrated.
##[error]Error: Networking validation for the Function app and Storage account has failed. Please review all error messages.

Aditional info

TL;DR;

AzureFunctionApp@2 fails with "Function app has private endpoint(s). But you are not running this pipeline from a self-hosted agent that has access to the Functions App. Relevant documentation: https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/agents?view=azure-devops&tabs=browser#install " even when "AzureCloud" is allowed access to SCM site.

Note that Pipelines are able to talk to the Function app as folders and logs are created by Pipelines in the /site/deployments folder.

Using AzureRmWebAppDeployment@4 works correctly.

Details

Create an Azure Function with the following security restrictions (Bicep syntax)

  properties: {
    …
    siteConfig: {
    …
      scmIpSecurityRestrictions: [
        {
          name: 'Azure DevOps'
          action: 'Allow'
          priority: 600
          tag: 'ServiceTag'
          ipAddress: 'AzureCloud'
        }
      ]
      scmIpSecurityRestrictionsDefaultAction: 'Deny'
      ipSecurityRestrictionsDefaultAction: 'Deny'
      scmIpSecurityRestrictionsUseMain: false
      ipSecurityRestrictions: [ ]
    }
    publicNetworkAccess: 'Enabled'
  }

Then add a Private EndPoint

When you try to deploy like this from an Azure Pipeline:

            - task: AzureFunctionApp@2
              displayName: "Deploy Management Subsystem to ${{ parameters.mgtSubsysAppName }}"
              inputs:
                azureSubscription: ${{ parameters.azureSubscription }}
                appType: functionAppLinux
                appName: ${{ parameters.mgtSubsysAppName }}
                ResourceGroupName: ${{ parameters.resourceGroupName }}
                package: "$(System.ArtifactsDirectory)/*.zip"
                runtimeStack: 'NODE|18'

You get the following result:

Starting: Deploy Management Subsystem to obxx-as 
============================================================================== 
Task : Azure Functions Deploy 
Description : Update a function app with .NET, Python, JavaScript, PowerShell, Java based web applications 
Version : 2.221.105 
Author : Microsoft Corporation 
Help : https://aka.ms/azurefunctiontroubleshooting 
============================================================================== 
Got service connection details for Azure App Service:'obxxxx-as' 
(node:1611) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead. 
##[error]Function app has private endpoint(s). But you are not running this pipeline from a self-hosted agent that has access to the Functions App. Relevant documentation: https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/agents?view=azure-devops&tabs=browser#install 
NOTE: Function app is VNet integrated. 
##[error]Error: Networking validation for the Function app and Storage account has failed. Please review all error messages. 
Successfully updated deployment History at https://obxxx-as.scm.azurewebsites.net/api/deployments/331686839307234 
App Service Application URL: https://obxxx-as.azurewebsites.net 
Finishing: Deploy Management Subsystem to obxxx-as 

Importantly, Pipelines does manage to create a deployments folder on the Function App server and create a log file, with the content "Deployment failed."

Using the following instead - which is not recommended - successfully deploys the app:

            - task: AzureRmWebAppDeployment@4
              displayName: "Deploy Management Subsystem to ${{ parameters.mgtSubsysAppName }}"
              inputs:
                appType: functionAppLinux
                ConnectionType: AzureRM
                ConnectedServiceName: ${{ parameters.azureSubscription }}
                ResourceGroupName: ${{ parameters.resourceGroupName }}
                WebAppName: ${{ parameters.mgtSubsysAppName }}
                packageForLinux: "$(System.ArtifactsDirectory)/*.zip"

[knn-halfspace]

We experience the same error here

[jj-halfspace]

Another way to use the work around when using the GUI, is to use the 'Azure App Service Deploy' task and select the App Service type to be Function App on Linux / Function App on Windows. Note that the system clearly tells you that this is not recommended on at least two occasions; when making the App Service type selection and when running the task on the agent, it will give the following message:

[warning]Recommendation: Use Azure Functions Task to deploy Function app.

Nonetheless the Function App is deployed successfully.

[TaylorWillis]

We are running into the same problem. Any updates on this?

[deepakwipfli]

We are also facing same problem. I tried deploying through @jj-halfspace 's step but no luck

[gesnaud]

We ran into same situation! thanks @jj-halfspace

[raeeeeeeeeeee]

May I know when this will be fixed? We encounter same issue now @FinVamp1 thanks

[sdzunenko]

We are experiencing the same. Please note: switching back to version@1 works fine, and just left same message in logs but with the WARNING statement.

Is this going to be fixed?

[teknofobia]

Just wanted to add that I was having the same problem when trying to deploy to a function app I set into a private vNet.

I have it with a private link and vnet integration, but configured to allow certain IP's through. I have it set to deny all and I added the Tags AzureDevOps and AzureCloud on the allow list in the tools site (because I'm using the shared build agents and not hosting my own).

This worked for deploying a regular app service, and should work for the Function deploy, but does not. I also have my storage account on the same vnet in a different subnet with access granted and the Function Deploy errors because it claims it cannot connect to the storage account.

Switching to an App service deploy did deploy my code as @jj-halfspace recommended.

[stateofthearb]

Come on Microsoft! We need cloud hosted VNET integrated agents already like GitHub does.

[JBAtDevoteam]

Is there any update on this issue? Experiencing it too, and I would like to know if a fix can be expected, or if a workaround is needed.

[FinVamp1]

Hello, it's on the backlog for the task and we need to figure out what the right thing is to do. No ETA at present.

[roundbatman]

What is the recommended workaround as this does not seem to be on a high priority? Seems like switching back to version 1 or using the app services deployment is working.

[TaylorWillis]

We utilized the deployment slots to get around this. Since the issue is that you cannot directly deploy to a slot that is inside the vnet, we added some a script to our pipeline that spins up a new deployment slot outside of the vnet, deploy the new code to it, swap the new deployment slot with the old one, then delete the old one.

This might not work for every use case, but it solved the problem for us.

[unix68]

Workaround using AzureRmWebAppDeployment@4 (as outlined by the original poster) also worked for me. Nevertheless I'd appreciate if this is fixed - or a statement is provided why MS decided to block deployments from MSHAs for private endpoint enabled functions. I think a warning instead of an error here would be enough. But I might miss some implications.

Source code reference: https://github.com/microsoft/azure-pipelines-tasks/blob/b6464a4cd567e9c8121337f338a2b54d0999a35d/Tasks/AzureFunctionAppV2/operations/AzureAppServiceUtilityExt.ts#L215