Isolated function cannot run with .NET 8 when Microsoft.Azure.Functions.Worker.Extensions.ServiceBus updated to 5.14.0 or newer

[toth-bence-nxbev]

Description

In an Isolated Function we use the Microsoft.Azure.Functions.Worker.Extensions.ServiceBus package with version 5.13.0, when we update the package and publish to Azure, the Function shows an error and does not start.

Steps to reproduce

Updating Microsoft.Azure.Functions.Worker.Extensions.ServiceBus from 5.13.0 to 5.14.0 or newer version causes the following error after publish:

Microsoft.Azure.WebJobs.Script: Error configuring services in an external startup class. Grpc.AspNetCore.Server: Could not load file or assembly 'Microsoft.AspNetCore.Routing, Version=7.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The system cannot find the file specified.

We reproduced this issue in a barebone project. Just by updating this one package we can reproduce the issue.

[dvdgutzmann]

We were also seeing this exact Exception with a newly deployed .NET 8 isolated function utilizing Microsoft.Azure.Functions.Worker.Extensions.ServiceBus in version 5.16.0 . Downgrading to 5.13.0 also fixed this issue for us for now.

[szathmari-martin-nxcsi]

The same exception was visible in the Azure Portal for us in a completely new empty project created in and published from Visual Studio 2022. Any version from 5.14.0 and up simply breaks the startup.

[imranmomin]

I may not be the expert.

But I believe it is common issue for Isolated function as those libraries are sometime not present in .azurefunctions folder. This folder is copied from the WorkerExtensions.csproj output path buildout\bin

The workaround I took is to manually copy the library into the .azurefunctions folder.

 <!-- hack: copy the dlls to .azurefunctions folder -->
 <ItemGroup>
     <MissingFiles Include="$(OutputPath)Name_Of_The_Library.dll"/>
 </ItemGroup>
 <Target Name="CopyToAzureFunctions" AfterTargets="Build">
     <Copy SourceFiles="@(MissingFiles)" DestinationFolder="$(OutputPath).azurefunctions" ContinueOnError="false"/>
 </Target>
 <Target Name="CopyFilesToPublish" AfterTargets="AfterPublish">
     <Copy SourceFiles="@(MissingFiles)" DestinationFolder="@(MissingFiles->'$(PublishDir).azurefunctions')" ContinueOnError="false"/>
 </Target>

[fabiocav]

@imranmomin the issue above seems a bit different, though, I'm curious to better understand what you're seeing, how often and with what configuration. If you can open a separate issue with those details, that would be great!

@tthbennex the issue you ran into seems like a problem with the TFM used for the extensions project. What version of the SDK are you using? Is this a failure you see locally as well, or when running in a CI/CD environment?

[toth-bence-nxbev]

@fabiocav We could reproduce the issue both when trying to publish from a local environment, and through a CI/CD environment (TeamCity) The exact version of the SDK I'm not sure I can tell you, cause we had an unrelated issue since then and I changed my SDK as a consequence of that, but I think it was 8.0.101

[imranmomin]

@fabiocav

We’ve built a custom isolated trigger library using .NET 6, which references Microsoft.Extensions.DependencyMode, Version=6.0.0.0.

The Azure function operates on .NET 8. When we build the project, we notice that the Microsoft.Extensions.DependencyMode library is missing from the .azurefunctions directory, even though it’s present in the extension project used to build the isolated trigger and also in the Release directory

An error occurred while setting up services in an external startup class. Custom.Isolated.Trigger: The system was unable to load the file or assembly ‘Microsoft.Extensions.DependencyModel, Version=6.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60’. The specified file could not be found.

However, when we manually copy the library, the Azure function operates as expected

https://github.com/Azure/azure-functions-dotnet-worker/issues/2270#issuecomment-1934979681

[igor-henriques]

I'm having the same issue here with a function created from the Service Bus NET 8 Isolated template. I can execute locally without any issue, but when I deploy the function (I'm doing through pipeline with action version @2), it never startups.

My error:

Microsoft.Azure.WebJobs.Script.ExternalStartupException : Error configuring services in an external startup class. ---> System.IO.FileNotFoundException : Could not load file or assembly 'Microsoft.AspNetCore.Routing, Version=7.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The system cannot find the file specified

My deploy stage:

- stage: Deploy
  displayName: Deploy stage
  dependsOn: Build
  condition: succeeded()

  jobs:
  - deployment: Deploy
    displayName: Deploy
    environment: 'development'
    pool:
      name: 'SelfHostedAgentPool'
      vmImage: 'ubuntu-latest'

    strategy:
      runOnce:
        deploy:
          steps:
          - task: AzureFunctionApp@2
            displayName: 'Azure functions app deploy'
            inputs:
              azureSubscription: '$(azureSubscription)'
              appType: functionApp
              appName: $(functionAppName)
              package: '$(Pipeline.Workspace)/drop/$(Build.BuildId).zip'

The thing is, I don't even have this .dll in my output folder, although I have the package as a dependency from Microsoft.Azure.WebJobs.Extensions.Http (3.2.0), which in turn is a dependency of Microsoft.NET.Sdk.Functions (4.0.0).

The funny thing is that even in the project that dependency in the version I have installed (forced) is shown as deprecated:

and guess what, I can't downgrade to a non-deprecated version (what?!) because it is a dependency of a major package, and I can't also see that dll in my output folder as well.

Plus: the template offered by VS (2022) didn't even include the package that is causing the issue here on this topic: Microsoft.Azure.Functions.Worker.Extensions.ServiceBus

All packages I have now are:

<PackageReference Include="Azure.Messaging.ServiceBus" Version="7.17.5" />
<PackageReference Include="Microsoft.AspNet.WebApi.Client" Version="6.0.0" />
<PackageReference Include="Microsoft.Azure.Functions.Extensions" Version="1.1.0" />
<PackageReference Include="Microsoft.Azure.WebJobs.Extensions.ServiceBus" Version="5.16.1" />
<PackageReference Include="Microsoft.Extensions.Configuration.UserSecrets" Version="8.0.0" />
<PackageReference Include="Microsoft.NET.Sdk.Functions" Version="4.4.0" />
<PackageReference Include="Microsoft.VisualStudio.Azure.Containers.Tools.Targets" Version="1.20.1" />
<PackageReference Include="Polly" Version="8.4.0" />
<PackageReference Include="System.Configuration.ConfigurationManager" Version="8.0.0" />
<PackageReference Include="Twilio" Version="7.1.1" />

So, indeed, the root cause here, at least for me, is Microsoft.NET.Sdk.Functions, which I can't just downgrade because all non-deprecated versions still has that same dependency chain that end up needing the same Microsoft.AspNetCore.Routing, Version=7.0.0 package

[szathmari-martin-nxcsi]

This is still a real issue that causes problems on production environments. It's hard to believe that so far no attention was paid towards something that is clearly not a sporadic but a reproducible failure.