Flex consumption with dotnet fails

davidobrien1985 commented 3 months ago

Attempting to deploy a dotnet8 app to a flex consumption Function App:

  publish-flex:
    runs-on: windows-latest
    needs: build
    steps:
    - name: 'Download Build Artifacts'
      uses: actions/download-artifact@v4
      with:
        name: app-build
        path: output

    - name: 'Publish Azure Flex Functions'
      uses: Azure/functions-action@v1.5.2
      with:
        app-name: ${{ env.AZURE_FUNCTIONAPP_FLEX_NAME }}
        package: output
        publish-profile: ${{ secrets.PUBLISH_PROFILE }}
        sku: flexconsumption
        remote-build: true
        enable-oryx-build: false
        scm-do-build-during-deployment: false

This is the GitHub Actions output:

Successfully parsed SCM credential from publish-profile format.
Using SCM credential for authentication, GitHub Action will not perform resource validation.
Warning: Neither AzureWebJobsStorage nor AzureWebJobsStorage__accountName exist in app settings (from Kudu SCM site with publish-profile credential). Please ensure one of them is configured as it is critical for function runtime. For more information, please visit the function app settings reference page: https://docs.microsoft.com/en-us/azure/azure-functions/functions-app-settings#azurewebjobsstorage
Successfully acquired app settings from function app (with SCM credential)!
Will archive output into D:\a\_temp\temp_web_package_65[20](https://github.com/argos-au/argos/actions/runs/10193129493/job/28198133521#step:3:21)085252805692.zip as function app content
Will use Kudu https://<scmsite>/api/publish to deploy since Flex consumption plan is detected.
Will use parameter remote-build: true
Package deployment using One Deploy initiated.
Starting deployment pipeline.
[ValidationStep] starting.
[AppSettingValidation] starting.
[DeploymentStorageValidation] starting.
Validation completed
[SourcePackageUriDownloadStep] starting.
Zip package is present at /tmp/zipdeploy/a32def0f-6667-49a7-8855-9658271bdd3c.zip
[ExtractZipStep] starting.
Cleaning files in /tmp/zipdeploy/extracted
Extracted zip package in /tmp/zipdeploy/extracted
[OryxBuildStep] starting.
Running oryx build command: oryx build /tmp/zipdeploy/extracted -o /home/site/wwwroot --platform dotnet
Error: Couldn't detect a version for the platform 'dotnet' in the repo.
Error: Couldn't detect a version for the platform 'dotnet' in the repo.\n/opt/Kudu/Scripts/starter.sh oryx build /tmp/zipdeploy/extracted -o /home/site/wwwroot --platform dotnet
Failed during oryx build due to Error: Couldn't detect a version for the platform 'dotnet' in the repo.\n/opt/Kudu/Scripts/starter.sh oryx build /tmp/zipdeploy/extracted -o /home/site/wwwroot --platform dotnet
Deployment failed with Error: Error: Couldn't detect a version for the platform 'dotnet' in the repo.\n/opt/Kudu/Scripts/starter.sh oryx build /tmp/zipdeploy/extracted -o /home/site/wwwroot --platform dotnet
Error: Failed to deploy web package to Function App.
Error: Execution Exception (state: PublishContent) (step: Invocation)
Error:   When request Azure resource at PublishContent, oneDeploy : Failed to use D:\a\_temp\temp_web_package_6520085[25](https://github.com/argos-au/argos/actions/runs/10193129493/job/28198133521#step:3:26)2805692.zip as OneDeploy content
Error:     Package deployment using ZIP Deploy failed. Refer logs for more details.
Error: Deployment Failed!

I'm confused about two things:

The warning about the AzureWebJobsStorage app setting. We never had these on our apps and they always worked.
Error: Couldn't detect a version for the platform 'dotnet' in the repo. - Why is it running oryx if it's set to false?

davidobrien1985 commented 3 months ago

Is this being looked at?

cc @patelchandni ?

patelchandni commented 3 months ago

@davidobrien1985 - You have passed remoteBuild: true parameter. Therefore, there is remote build performed using oryx. If you do not want remote build, then change that parameter to false.

davidobrien1985 commented 3 months ago

@patelchandni I'm confused. This is to deploy to a Flex Consumption plan. According to the readme on this repo that is exactly the config we should pass.

Parameters only supported for Flex Consumption plan:

sku: For function app on Flex Consumption plan, set this to flexconsumption. You can skip this parameter for function app on other plans. If using RBAC credentials, then by default, GitHub Action will resolve the value for this paramter. But if using publish-profile, then you must set this for function app on Flex Consumption plan.
remote-build: For function app on Flex Consumption plan, enable build action from Kudu when the package is deployed onto the function app by setting this to true. For function app on Flex Consumption plan, do not set scm-do-build-during-deployment and enable-oryx-build. By default, this is set to false.

davidobrien1985 commented 2 months ago

@patelchandni are you still monitoring this?

I tried again using the exact parameters that the readme on this repo says I should use, but set remote-build to false now:

- name: 'Publish Azure Flex Functions EU'
      uses: Azure/functions-action@v1.5.2
      with:
        app-name: ${{ env.AZURE_FUNCTIONAPP_FLEX_NAME_EU }}
        package: output
        publish-profile: ${{ secrets.PUBLISH_PROFILE_EU }}
        sku: flexconsumption
        remote-build: false

That indeed finally uploaded the zip file to the container configured on the Consumption Flex Function App, but the Function App only shows WarmUp Function, not our app. Why is that? Do we need to build the app differently? Does the output package need to look differently?

The same package works without any issues on regular Function Apps. (see below)

- name: 'Publish Azure Premium Functions EU'
      uses: Azure/functions-action@v1.5.2
      with:
        app-name: ${{ env.AZURE_FUNCTIONAPP_NAME_EU }}
        package: output
        publish-profile: ${{ secrets.PUBLISH_PROFILE_PREMIUM_EU }}

frasermclean commented 2 months ago

I am getting the same "WarmUp" function displaying after deploying using this action. This Action was previously working for me successfully when deploying to a Flex Consumption plan. In my context, I am using RBAC to deploy with an azure/login@v1 action.

If I deploy manually from my desktop using VSCode, all the correct functions are visible in the portal.

Here are my workflow job steps:

      # Download artifact
      - name: Download artifact
        id: download-artifact
        uses: actions/download-artifact@v4
        with:
          name: ${{ inputs.jobsArtifactName }}

      # Login to Azure
      - name: Login to Azure
        uses: azure/login@v1
        with:
          tenant-id: ${{ secrets.AZURE_TENANT_ID }}
          client-id: ${{ secrets.AZURE_CLIENT_ID }}
          subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}

      # Publish application
      - name: Publish application
        uses: Azure/functions-action@v1
        with:
          app-name: ${{ needs.deploy_infra.outputs.functionAppName }}
          package: ${{ steps.download-artifact.outputs.download-path }}

davidobrien1985 commented 2 months ago

@frasermclean does that really deploy a Flex Consumption app? Looks like it's missing the sku parameter?

frasermclean commented 2 months ago

@frasermclean does that really deploy a Flex Consumption app? Looks like it's missing the sku parameter?

According to the docs, the action determines the correct SKU as I am using RBAC auth.

davidobrien1985 commented 2 months ago

@patelchandni any idea why this is happening? Now two people get the same incorrect outcome.

mangzee commented 2 months ago

Unfortunately, I have the same issue, I tried with remoteBuild: false and the deployment succeeded but I do not see the functions.

davidobrien1985 commented 2 months ago

@patelchandni or anybody else from Microsoft? Anybody watching this?

nzthiago commented 2 months ago

We are still investigating @davidobrien1985's app through a support case. We mitigated the issue to a certain degree by adding WEBSITE_USE_PLACEHOLDER_DOTNETISOLATED app setting and setting a value of 0. Unfortunately, it is not possible to tell if this is the same issue @mangzee, we would need a support ticket to be able to investigate your app as well, as the app is deploying but failing to show the functions.

mangzee commented 2 months ago

We are still investigating @davidobrien1985's app through a support case. We mitigated the issue to a certain degree by adding WEBSITE_USE_PLACEHOLDER_DOTNETISOLATED app setting and setting a value of 0. Unfortunately, it is not possible to tell if this is the same issue @mangzee, we would need a support ticket to be able to investigate your app as well, as the app is deploying but failing to show the functions.

@nzthiago @davidobrien1985 I'm sorry for not getting back to you sooner, but I could manage to get it to work. I'm not sure which of the below got it working, but these are steps performed

Set WEBSITE_WEBDEPLOY_USE_SCM = false on the function env variables.
And this was the YAML used in github actions

name: Deploy DotNet project to Azure Function App

on:
  push:
    branches: ["main"]

env:
  AZURE_FUNCTIONAPP_NAME: 'testfunc'   # set this to your function app name on Azure
  AZURE_FUNCTIONAPP_PACKAGE_PATH: 'Src/Functions/TestFunc'       # set this to the path to your function app project, defaults to the repository root
  DOTNET_VERSION: '8.0.x'                   # set this to the dotnet version to use (e.g. '2.1.x', '3.1.x', '5.0.x')

jobs:
  build-and-deploy:
    runs-on: ubuntu-latest
    environment: dev
    steps:
    - name: 'Checkout GitHub Action'
      uses: actions/checkout@v4

    # If you want to use Azure RBAC instead of Publish Profile, then uncomment the task below
    # - name: 'Login via Azure CLI'
    #   uses: azure/login@v1
    #   with:
    #     creds: ${{ secrets.AZURE_RBAC_CREDENTIALS }} # set up AZURE_RBAC_CREDENTIALS secrets in your repository

    - name: Setup DotNet ${{ env.DOTNET_VERSION }} Environment
      uses: actions/setup-dotnet@v4
      with:
        dotnet-version: ${{ env.DOTNET_VERSION }}

    - name: 'Resolve Project Dependencies Using Dotnet'
      shell: bash
      run: |
        pushd './${{ env.AZURE_FUNCTIONAPP_PACKAGE_PATH }}'
        dotnet build --configuration Release --output ./output
        popd

    - name: 'Run Azure Functions Action'
      uses: Azure/functions-action@v1
      id: fa
      with:
        app-name: ${{ env.AZURE_FUNCTIONAPP_NAME }}
        package: '${{ env.AZURE_FUNCTIONAPP_PACKAGE_PATH }}/output'
        publish-profile: ${{ secrets.AZURE_FUNCTIONAPP_PUBLISH_PROFILE }} # Remove publish-profile to use Azure RBAC
        sku: flexconsumption
        remote-build: false

bbevers2 commented 1 month ago

For dotnet packages, make sure that WEBSITE_RUN_FROM_PACKAGE does not exist in your Azure Environment variables for deployments with

        sku: flexconsumption
        remote-build: false

Otherwise, you may receive 404

David-M-Baxter commented 1 month ago

FYI, I've tried publish-profile without better success. Eventually i came to this:

name: Deploy Azure Functions App uses: Azure/functions-action@v1 with: app-name: ${{ steps.tfoutputs.outputs.functionapp_name }} package: ohana-server-${{ env.VERSION }}.zip

Here's the output:

Using RBAC for authentication, GitHub Action will perform resource validation. Successfully acquired site configs from function app! Detected function app sku: Consumption Successfully acquired app settings from function app (RBAC)! Detected function app language: Node Will directly deploy ohana-server-1.10.0-build.1272.zip as function app content. Will use WEBSITE_RUN_FROM_PACKAGE to deploy since RBAC is detected and your function app is on Linux Consumption. Using AzureWebJobsStorage for Blob access. Package Url will use SAS. Sync Trigger call was successful.

davidobrien1985 commented 1 month ago

I get 404 deployment errors on basically every second deployment. I rerun the failed jobs, all good.

Get Outlook for Androidhttps://aka.ms/AAb9ysg

From: David Maugrion @.> Sent: Saturday, October 19, 2024 4:50:23 AM To: Azure/functions-action @.> Cc: David O'Brien @.>; Mention @.> Subject: Re: [Azure/functions-action] Flex consumption with dotnet fails (Issue #245)

FYI, I've tried publish-profile without better success. Eventually i came to this:

name: Deploy Azure Functions App uses: @.*** with: app-name: ${{ steps.tfoutputs.outputs.functionapp_name }} package: ohana-server-${{ env.VERSION }}.zip

Here's the output:

Using RBAC for authentication, GitHub Action will perform resource validation. Successfully acquired site configs from function app! Detected function app sku: Consumption Successfully acquired app settings from function app (RBAC)! Detected function app language: Node Will directly deploy ohana-server-1.10.0-build.1272.zip as function app content. Will use WEBSITE_RUN_FROM_PACKAGE to deploy since RBAC is detected and your function app is on Linux Consumption. Using AzureWebJobsStorage for Blob access. Package Url will use SAS. Sync Trigger call was successful.

— Reply to this email directly, view it on GitHubhttps://github.com/Azure/functions-action/issues/245#issuecomment-2422961078, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACBIQGLYLZKMOEUENG7G4WDZ4FC57AVCNFSM6AAAAABLZ53EBCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMRSHE3DCMBXHA. You are receiving this because you were mentioned.Message ID: @.***>

ablyler commented 2 weeks ago

If you are downloading and artifact, it could it be related to this issue: https://github.com/Azure/azure-functions-dotnet-worker/issues/1240

In the actions/upload-artifact step, the include-hidden-files: true line is the key. Without it the .azurefunctions directory isn't included in the artifact, which causes the function to not deploy correctly.

anandsowm commented 1 week ago

For those deploying to a flex consumption plan app from Azure DevOps pipelines, make sure to check the checkbox "Is Function app on Flex Consumption plan", only then the dotnet deployment succeeds.

john-manktelow commented 1 day ago

Any errors in the function's entry point seem to lead to this state, and because deployment spins up the function app to get its list of triggers, it all looks like issues with deployment rather than our own code.

Compounding this (because Flex Consumption functions have done away with app logs and log streaming) there seems to be no way to actually diagnose what is causing the problem.

To reproduce:


// Program.cs in an otherwise standard function app

var builder = FunctionsApplication.CreateBuilder(args);

builder.ConfigureFunctionsWebApplication();

builder.Services
     .AddApplicationInsightsTelemetryWorkerService()
     .ConfigureFunctionsApplicationInsights();

// Any exception during startup leads to 'WarmUp' and looks like a deployment issue
throw new InvalidOperationException("You will never see this message");

builder.Build().Run();

Deploying this (using Core Tools here, but I get the same end-state using the action):

func azure functionapp publish <flex-function-name>

gives an 'operation failed successfully' sort of message:

[...]
Uploaded blob successfully.
Uploaded package to storage blob. Deployment is partially successful from here.
[RemoveWorkersStep] starting.
RemoveAllWorkers, statusCode = NoContent
Reset all workers was successful.
[SyncTriggerStep] starting.
Waiting 60 seconds for the workers to recycle with deployed content.
[CleanUpStep] starting.
Cleaned the source packages directory.
Cleaned the result artifact directory.
Finished deployment pipeline.
Checking the app health...Deployment was successful but the app appears to be unhealthy, please check the app logs.

(note that the SyncTriggerStep never completes)

In the portal, the flex function app now has a single HTTP trigger called 'WarmUp'. In App Insights, there are no Exceptions, and the exception thrown during startup is not present in the traces. In the deployment logs, the SyncTriggerStep is shown to be failing with Bad Gateway:

{"AspNetCoreEnvironment":"Production","MessageTemplate":"FunctionHostSyncTrigger, statusCode = BadGateway","SourceContext":"Kudu.Legion.DeploymentSteps.SyncTriggerStep","deploymentId":"4f9dfe77-86e0-4f17-b8a3-a743cd0cc8eb"}

This has cost a lot of time - can we please have the log stream back for flex functions (and include the app's output during deployment)?

Or, if it has to all be in app insights, can there be some guidance on how to find these startup errors during deployment?

Can the deployment log include the actual exception information rather than Bad Gateway?

As it is there is an unobservable gap between deployment logs, and the tracing around a function's triggered execution.

Azure / functions-action

Flex consumption with dotnet fails #245