Azure / azure-functions-vs-build-sdk

MSBuild task for Azure Functions
MIT License
96 stars 65 forks source link

Dependency mismatch when using Entity Framework Core 5.0 from Functions app #472

Closed peterwurzinger closed 3 years ago

peterwurzinger commented 3 years ago

I tried to migrate one of our apps (Azure Functions v3, .NET Core 3.1) from using Entity Framework Core 3.1 to Entity Framework Core 5.0 (RC 1).

Even after several hours of trying, i couldn't get around this error:

A host error has occurred during startup operation '52d3462b-75d1-44d6-ab46-81e5210f4fc1'.
TestFunctionWithEfCore5: Could not load file or assembly 'Microsoft.Extensions.DependencyInjection.Abstractions, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. System cannot find the file specified.

Did I miss an announcement, that this scenario isn't supported? Since Entity Framework Core only depends on netstandard2.1 I expected it to be usable from a v3/netcoreapp3.1 app.

EDIT: I'd happily provide a sample to reproduce the error, if needed.

skvark commented 3 years ago

I might be wrong but it looks like this SO answer describes the root cause of this issue: https://stackoverflow.com/a/64006926/5795428

Edit:

Same issue reported around different repos:

peterwurzinger commented 3 years ago

@skvark Absolutely, thanks for pointing me there. I can't really tell why I didn't find the SO thread last week, since it covers the same question.

EDIT: I'll keep the issue open though, since I think that the accepted answer on SO is not correct.

skvark commented 3 years ago

Yeah I think this issue needs to be solved somewhere. It prevents DI usage with Azure Functions and EF Core 5. Solution is to stay at older EF Core version. Imho should be pretty high priority to get this fixed since EF Core 5 (and .NET 5) will be out in couple of months and probably most of us would like to use the latest versions also with Azure Functions.

anthonychu commented 3 years ago

@peterwurzinger could you please provide a repro?

peterwurzinger commented 3 years ago

@anthonychu Absolutely. It's located at https://github.com/peterwurzinger/azure-functions-ef-core-5-repro

peterwurzinger commented 3 years ago

To share additional findings of debugging that one: When creating the emptiest possible Azure Functions v3 Project, and executing it using the Azure Functions CLI, there are actually quite a lot of 3.1 assemblies that have already been loaded by func.exe. Even if it's because the func.exe application needs them to work, or on purpose for whichever reason. But I'm pretty sure, that this has something to do with it...

Following are the Microsoft.Extensions.* - packages, that have been loaded when simply trying to host an empty Azure Functions App with its CLI. Name Version Process
Microsoft.Extensions.WebEncoders.dll 3.100.820.42004 [2452] func.exe
Microsoft.Extensions.Primitives.dll 3.100.820.42004 [2452] func.exe
Microsoft.Extensions.Options.dll 3.100.820.42004 [2452] func.exe
Microsoft.Extensions.Options.ConfigurationExtensions.dll 3.100.820.42004 [2452] func.exe
Microsoft.Extensions.ObjectPool.dll 3.100.820.42004 [2452] func.exe
Microsoft.Extensions.Logging.EventSource.dll 3.100.820.42004 [2452] func.exe
Microsoft.Extensions.Logging.dll 3.100.820.42004 [2452] func.exe
Microsoft.Extensions.Logging.Debug.dll 3.100.820.42004 [2452] func.exe
Microsoft.Extensions.Logging.Console.dll 3.100.820.42004 [2452] func.exe
Microsoft.Extensions.Logging.Configuration.dll 3.100.820.42004 [2452] func.exe
Microsoft.Extensions.Logging.Abstractions.dll 3.100.820.42004 [2452] func.exe
Microsoft.Extensions.Localization.Abstractions.dll 3.100.820.42004 [2452] func.exe
Microsoft.Extensions.Hosting.dll 3.100.820.42004 [2452] func.exe
Microsoft.Extensions.Hosting.Abstractions.dll 3.100.820.42004 [2452] func.exe
Microsoft.Extensions.FileSystemGlobbing.dll 3.100.820.42004 [2452] func.exe
Microsoft.Extensions.FileProviders.Physical.dll 3.100.820.42004 [2452] func.exe
Microsoft.Extensions.FileProviders.Abstractions.dll 3.100.820.42004 [2452] func.exe
Microsoft.Extensions.DependencyInjection.dll 3.100.820.42004 [2452] func.exe
Microsoft.Extensions.DependencyInjection.Abstractions.dll 3.100.820.42004 [2452] func.exe
Microsoft.Extensions.Configuration.Json.dll 3.100.820.42004 [2452] func.exe
Microsoft.Extensions.Configuration.FileExtensions.dll 3.100.820.42004 [2452] func.exe
Microsoft.Extensions.Configuration.EnvironmentVariables.dll 3.100.820.42004 [2452] func.exe
Microsoft.Extensions.Configuration.dll 3.100.820.42004 [2452] func.exe
Microsoft.Extensions.Configuration.CommandLine.dll 3.100.820.42004 [2452] func.exe
Microsoft.Extensions.Configuration.Binder.dll 3.100.820.42004 [2452] func.exe
Microsoft.Extensions.Configuration.Abstractions.dll 3.100.820.42004 [2452] func.exe
Microsoft.Extensions.Caching.Memory.dll 3.100.820.42004 [2452] func.exe
Microsoft.Extensions.Caching.Abstractions.dll 3.100.820.42004 [2452] func.exe
anthonychu commented 3 years ago

Thanks @peterwurzinger!

@fabiocav @brettsam Anything that can be done to make this work in Functions V3?

fabiocav commented 3 years ago

Looking at your setup would help. What framework are you targeting? The dependency that failed above is to a .NET 5.0 assembly, which is not be supported in 3.0 yet.

I you can share a quick project on GitHub, we should be able to provide some guidance here.

@peterwurzinger yes, 3.1 dependencies are used by the host running your functions, hence the current dependency on the framework target and the assemblies you're seeing.

peterwurzinger commented 3 years ago

@fabiocav

I you can share a quick project on GitHub, we should be able to provide some guidance here.

I already provided a repository to reproduce the error (see above): https://github.com/peterwurzinger/azure-functions-ef-core-5-repro

fabiocav commented 3 years ago

Ah! Thank you! Somehow, I managed to miss that. We'll take a closer look here.

FitzyCodesThings commented 3 years ago

Howdy! Just adding my thoughts here.

This is a major issue for us at the moment. We can work around it by using a WebJob in place of our Function, but this is non-ideal for a number of reasons.

Considering that the EF Core team is on RC 2 and encouraging use of the candidate, any updates on a timeframe to get this game-ender addressed in Functions?

Thanks!

fabiocav commented 3 years ago

@FitzyCodesThings this is currently how the managed runtime works, the framework/.NET dependencies are tied to the version it supports.

Support for .NET 5.0 and .NET 5 dependencies is being actively worked on, but it is something that will ship after .NET 5.0 RTMs

Chryogenic commented 3 years ago

I am also having this issue while trying to run a v3 Azure function with dependency injection and Entity Framework Core 5.0 (RC 2).

RichMercer commented 3 years ago

This looks like a barrier to my application adopting .net 5.0 as I rely on dependency injection in Azure Functions. I'm not using EF , but I assume this would affect anyone using DI in Functions as I'm seeing the same error?

peterwurzinger commented 3 years ago

@RichMercer As far as I'm concerned I think this basically affects every package, that the Func CLI uses internally (see above), and therefore brings into the App Domain (uuh, I said the "A" word) of the hosted Function App.

RichMercer commented 3 years ago

Support for .NET 5.0 and .NET 5 dependencies is being actively worked on, but it is something that will ship after .NET 5.0 RTMs

Is there some idea on timeline for .NET 5.0 support to assist with our planning? We’re unable to move to .NET 5.0 until this is released.

Schuer84 commented 3 years ago

Also running into this issue. Azure functions v3 (.net core 3.1) in combination with EFCore 5.0.

pinkfloydx33 commented 3 years ago

We have several shared libraries that we publish internally that we consume from our functions. Those libraries have refs to various MS.Extensions packages. I can't update my shared code to use the new 5.0 packages because it then breaks the Azure Functions due to the assembly version issue.

Obviously I'd increase the major version of my internal packages but it makes it difficult to provide updates that the function apps would need (effectively pinning them).

Is there any sort of work around for this scenario? Binding redirects, preview versions or anything else? I'm thinking for now I could multi-target netstandard2.1/net5.0 and just reference the older libs for netstandard but that's really not the approach I'd like to take since they are supposedly compatible

hasandogu commented 3 years ago

This is affecting all our internal libraries as well. We upgraded to EF Core 5 and realized that we are affected by this issue in all our Functions. Is there any workaround to this at all?

AartBluestoke commented 3 years ago

there can not be any work around for this, except for a new version of azure functions: (copied from my prior response at https://github.com/Azure/azure-functions-host/issues/6893#issuecomment-726386105 )

Consequence: if you need a library that is a direct or indirect dependency of azure functions you can only use the version that is loaded. This is a case where azure functions is incorrectly reporting its hard dependencies.

eg: if you are using azure functions 3.0.7, then you must use a library binary compatible with dependencyInjection.abstractions 2.1.0 When your code (eg if you have a asp.netcore dependencency) asks for something that is only found in Abstractions >=5.0 the runtime (correctly) responds "sorry, that function isn't there", because all it has is the 2.1.0 dll in memory.

image

Therefore the many tickets that will be raised that are like this can only be solved by upgrading azure functions to a newer version which has a dependency on the 5.x line of the abstractions.

In this particular case, if azure functions used dependencyinjection.abstractions 5.0, then it would be fine because that library appears to be backwards compatible so the reverse error doesn't occur (no runtime failure if you expect 2.1 and actually get 5.0).

If that is not true for all libraries in common between azure functions and the libraries your code uses, then you will be pinned in terms of your dependencies and azure functions versions.

hasandogu commented 3 years ago

Appreciated the detailed response @hkzenetex. We will hold on upgrading all our projects to .NET 5 until this issue is resolved.

MyEidos commented 3 years ago

Can the new feature "dotnet-isolated Functions runtime" help with it? Any documentation? https://github.com/Azure/azure-functions-core-tools/releases/tag/3.0.3160

AartBluestoke commented 3 years ago

I believe that is part of the solution so that a 5.0 dependent version of azure functions can run on top of the current 2.1 targeting functions. (there is no current intent to update the true runtime for azure functions to 5.0, as 5.0 isn't a LTS version) (ie, it will allow you to run .net as another 'foreign' language (such as python), decoupling from the underlying runtime versions) -- more info at https://github.com/Azure/azure-functions-host/issues/6674#issuecomment-738299872

DevOps801 commented 3 years ago

I'm not sure why Azure tech support gave me a link to this thread but in case my particular experience helps anyone getting started with C# Dot Net Core and Azure development, here goes. I was receiving the same error message but with 'Microsoft.Extensions.Primitives'. My workaround involved separating my one project into two resulting in both being in their own Function App "container" (for lack of a better word).

Prior to getting this error, I already had a Function App project deployed and working on Azure as expected. Handful of classes that use the Azure WebHook function. Each of these classes are Receivers for JSON data sent by an external 3rd party website. One Receiver for each form on that site. The activity involves receiving the JSON data, deserializing it into a class object for a sanity check then serializing it back to be stored into an Azure Message Queue. Then I started a different project in VS2019 to use the QueueTrigger Azure function. This activity involves consuming the item data in the queue, deserialize into class object, connect and authenticate to SharePoint 365 website, using CSOM, and save it into a specific list. It's when I thought I'd try just having both Receivers and Processors activities combined into one project one solution is where I went wrong and started getting the above error message. Azure's Tech support's suggestion to read all the stuff on this page was good information but, too much info so I just decided to go back to my original approach having both Receivers and Processors each in their own project and in their own Function App container on Azure. Both use the same resources, storage and key vault.

It's all working for me now as expected as it did in local debug mode. Tech support simply could have said, "Try separating your solution into their own Function App containers", would've saved me a lot of time. Took the entire day today to get it all sorted out.

ShaneFromFargo commented 3 years ago

Any update here? This is a pretty big issue, and is preventing us from moving forward on our Azure functions deployment.

Drew-E commented 3 years ago

Any update on this? How are people with EF Core 5 dependencies (i.e. shared libraries) supposed to use functions?

pmeems commented 3 years ago

A few weeks ago I started a new Blazor project using .NET5 and EFCore5. Everything works fine and now I want to create an Azure Function using a shared library and I get the errors mentioned earlier. Any timeline when this will be fixed is much needed.

For now, I will copy all of the needed EFCore code into the Function project. But that is a bad and dirty workaround.

Edit: It is even worse. I just copied my models but I use many-to-many relationships which EFCore5 handles simple but for EFCore3 I need to model the join tables and add a lot of Fluent code.

FitzyCodesThings commented 3 years ago

Any updates at all on this?? We're now MONTHS into .NET 5 / EF Core 5 / etc. (and already talking about .NET 6) and I still can't use Functions at all....

russharding commented 3 years ago

Any updates here? It would be a lot of work to have to downgrade my project to EF.Core 3.1 or more likely EF 6.x.

NameGrey commented 3 years ago

It's already April and no other solution than downgrading? We planned to start using Azure functions, but looks like we have to postpone our decision...

russharding commented 3 years ago

@NameGrey check this out: https://techcommunity.microsoft.com/t5/apps-on-azure/net-on-azure-functions-roadmap/ba-p/2197916

It solved my problem. It's a bit of a hassle and is the embodiment of "MVP", but it does work.

DevOps801 commented 3 years ago

@NameGrey check this out: https://techcommunity.microsoft.com/t5/apps-on-azure/net-on-azure-functions-roadmap/ba-p/2197916

It solved my problem. It's a bit of a hassle and is the embodiment of "MVP", but it does work.

Russ, interesting comment you sent to @NameGrey. In the case of my function app (two of them in their own deployed projects utilizing shared resource, i.e. storage aka queues), Azure support came back with this explanation for why my 2nd function app stopped working back in February: "We were checking the backend logs for the timeframes that you sent to me, and I found that there was an update on the host version of your function app. These updates are done by Azure to do maintenance, since function apps are hosted on a Platform as a Service cloud." I read the article you referred to @NameGrey and I was wondering if I should give that a try while Azure support continues its investigation/troubleshooting? If I read and understood the article correct, I just re-target my function app to use ".Net 5.0" (I don't see option for ".Net 5.0 (isolated)", recompile and redeploy to Azure. Correct?

Amthieu commented 3 years ago

Hi @anthonychu & @fabiocav ,

Does the Azure team have a plan to update Azure Functions dependencies to allow EF Core 5 to work?

jeffreypolk commented 3 years ago

I'm running into this issue as well using durable functions. According to the roadmap, this won't be fixed until .NET7 (yikes!). As a workaround, I include a console app with my function app. When I need to execute code that leverages EF Core (or any other conflicting dependency), my function method invokes the console app via System.Diagnostics.Process and waits for it to finish. This allows me to break free from the hosting process and run my code in its own process. I'm not running this in production (yet), but it seems to work just fine without issue. Does anyone see an issue with this approach?

mattd-its commented 3 years ago

I'm also running in to this issue. We have no plans to go to .NET 5 any time soon (so that as a solution isn't an option for us). We want to continue using .NET Core 3.1 and Azure Functions v3, with Entity Framework Core 5. For now v3 of EF Core is ok, but we really want the ability to use EF Core 5 with Azure Functions (and dependency injection).

I also have a working example showing that going from EF 3.1.15 to EF 5.x causes the following error. I can provide the example if still needed.

System.Private.CoreLib: Could not load file or assembly 'Microsoft.Extensions.Logging.Abstractions, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The system cannot find the file specified. Value cannot be null. (Parameter 'provider')

fabiocav commented 3 years ago

Support to .NET 5.0 (and 5.x dependencies) is available with the out-of-proc worker model, so it doesn't impact the SDK

We're shared the information about the roadmap and future plans here, so I'm closing this as I'm hoping this answers the questions, but if not, I'd be happy to clarify here.

russharding commented 3 years ago

@NameGrey check this out: https://techcommunity.microsoft.com/t5/apps-on-azure/net-on-azure-functions-roadmap/ba-p/2197916 It solved my problem. It's a bit of a hassle and is the embodiment of "MVP", but it does work.

Russ, interesting comment you sent to @NameGrey. In the case of my function app (two of them in their own deployed projects utilizing shared resource, i.e. storage aka queues), Azure support came back with this explanation for why my 2nd function app stopped working back in February: "We were checking the backend logs for the timeframes that you sent to me, and I found that there was an update on the host version of your function app. These updates are done by Azure to do maintenance, since function apps are hosted on a Platform as a Service cloud." I read the article you referred to @NameGrey and I was wondering if I should give that a try while Azure support continues its investigation/troubleshooting? If I read and understood the article correct, I just re-target my function app to use ".Net 5.0" (I don't see option for ".Net 5.0 (isolated)", recompile and redeploy to Azure. Correct?

Hi @DevOps801 sorry I missed this and I hope you were able to get your function up and running.

To answer your question, by following that article and retargeting to .NET 5, as well as setting your FUNCTIONS_WORKER_RUNTIME to "dotnet-isolated" you should find that the project will start to work.

I would definitely follow the linked article carefully. The solution is not the most user friendly; but it does work.

marklagendijk commented 3 years ago

I found this blog post about upgrading function apps to .NET 5. It contained quite a few instructions that were missing elsewhere.

hansmbakker commented 3 years ago

I found this blog post about upgrading function apps to .NET 5.

@marklagendijk that only works if you're willing to switch to the isolated dotnet worker instead of using the Functions runtime hosted in Azure - which has different pros/cons so not for everybody this isolated dotnet worker is better

marklagendijk commented 3 years ago

@hansmbakker that is correct, these are described in the announcement blog post that was mentioned earlier:

Pros:

A .NET 5 function app runs in an isolated worker process. Instead of building a .NET library loaded by our host, you build a .NET console app that references a worker SDK. This brings immediate benefits: you have full control over the application’s startup and the dependencies it consumes. The new programming model also adds support for custom middleware which has been a frequently requested feature.

Cons:

While this isolated model for .NET brings the above benefits, it’s worth noting there are some features you may have utilized in previous versions that aren’t yet supported. While the .NET isolated model supports most Azure Functions triggers and bindings, Durable Functions and rich types support are currently unavailable. Take a blob trigger for example, you are limited to passing blob content using data types that are supported in the out-of-process language worker model, which today are string, byte[], and POCO. You can still use Azure SDK types like CloudBlockBlob, but you’ll need to instantiate the SDK in your function process.

marklagendijk commented 3 years ago

Another important resource is the Azure Functions .NET Worker repository.