Azure / azure-functions-host

The host/runtime that powers Azure Functions
https://functions.azure.com
MIT License
1.94k stars 442 forks source link

.NET Core 2.2.1 + EF Core 2.2.1 causes load errors #4006

Closed JohnGalt1717 closed 4 years ago

JohnGalt1717 commented 5 years ago

Repro steps

Create a jobs project in vs.net and link it to an EF Core Data Class library that has a context that implements IdentityContext.

Get a context in your function something like this:

            var dbBuilder = new DbContextOptionsBuilder<dbCTxName>();

            var dbOptions = dbBuilder.UseLoggerFactory(loggerFactory).UseSqlServer(configuration["ConnectionStrings:DBName"], x =>
            {
                x.EnableRetryOnFailure(2);
            }
            ).EnableSensitiveDataLogging((Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT") ?? "development").ToLower() == "development")
            .Options as DbContextOptions<dbCTxName>;

            return new dbCTxName(dbOptions);

It will error on: var dbBuilder = new DbContextOptionsBuilder(); with a type loader error.

Expected behavior

Azure Functions and the function host should always just work no matter what version of .NET core you're using and should always be synchronized with all updates of .NET Core that come out otherwise security issues could be left open for months (last time was over a month from 2.1 to 2.2!)

Actual behavior

With 2.2.0 of EF Core nuget packages this will work fine. With 2.2.1 this will throw a type loader error.

Known workarounds

Downgrading nuget packages to 2.2.0 instead of 2.2.1 fixes the issue but there are security issues in 2.2.0

BernhardGa commented 5 years ago

The identity package https://www.nuget.org/packages/Microsoft.AspNetCore.Identity.EntityFrameworkCore/2.2.0 didn't received an update to 2.2.2 (or even 2.2.1) and fails.

fabiocav commented 5 years ago

@JohnGalt1717 the issue you've reported with 2.2.1 should be resolved, can you confirm?

JohnGalt1717 commented 5 years ago

We're on to 2.2.4 as of yesterday which Azure Function Host should have synchronized releases.

Is it working in 2.2.4 to your knowledge?

espray commented 5 years ago

@JohnGalt1717 - 2.2.1 is working for me. But, 2.2.2, 2.2.3 & 2.2.4 all failed for me when I logged #4290

  <PropertyGroup>
    <TargetFramework>netcoreapp2.2</TargetFramework>
    <LangVersion>latest</LangVersion>
    <AzureFunctionsVersion>v2</AzureFunctionsVersion>
  </PropertyGroup>
JohnGalt1717 commented 5 years ago

@espray @fabiocav This issue is about Azure Function's failure to synchronize releases to the .NET core releases and as a result, the Azure Functions runtime is always exposing it's customers to security vulnerabilities.

The bug will be fixed when:

  1. Same day releases of Azure Functions are released synchronized with the .net core releases. At least point releases.
  2. There is an obvious place to go that outlines exaclty what versions of Newtonsoft.Json and all other dependencies are required given that Azure Functions breaks hard with newer versions of libraries.
fabiocav commented 5 years ago

@JohnGalt1717 that's good feedback, and we do depend on the deployment cadence on App Service (Web Apps), which we run on top of. As of now, they are on 2.2.3, and we'll be moving to match with the next deployment. We'll be working on ways to make that process more visible and see what we can to do improve the turnaround.

There is an obvious place to go that outlines exaclty what versions of Newtonsoft.Json and all other dependencies are required given that Azure Functions breaks hard with newer versions of libraries.

I like the idea to list the supported runtime dependencies, which are essentially framework and WebJobs dependencies. (/cc @jeffhollan , not sure if docs would be the best place, given how often/quickly this might change, but it would be good to discuss). Are you experiencing issues with JSON.NET in V2? That should be correctly unified and you shouldn't see any runtime breaks with that dependency.

jeffhollan commented 5 years ago

I think having the x.y level of detail makes sense on docs (e.g. 2.2), but for the x.y.z level I'd prefer to point somewhere in GitHub or elsewhere that is providing that level -- ideally in an automated fashion -- but maybe even something like a GitHub wiki

JohnGalt1717 commented 5 years ago

@fabiocav It's fine if it's in sycn with App Service. The problem is that in my experience it very rarely is.

And there should be a way of overriding it if necessary. As an example, we use Docker functions. In this case it doesn't matter what the App Service is using, only what docker says to use. Same if you force install the .net core framework into the app service. (i.e. people wanting to test .net core 3)

As for dependencies, Newtonsoft.Json is on the latest 12.1 right now. I don't have a clue until they fail to execute that that's the version required. Even if func.exe could tell you would be a major plus. (/depends or something like that) I still assume that Functions aren't supporting 12.1 but who knows for sure?

There are a bunch of other Azure dependencies that don't work right too that you have to be very careful to get the right versions. The nuget package (Microsoft.NET.Sdk.functions should lock this stuff so that you can't install the wrong versions too. (and should list the dependancies. I see in nuget that it lists them, but it doesn't throw any errors etc. if you don't match exactly.

And all of these dependencies should be updated as soon as possible to the latest versions.

fabiocav commented 5 years ago

It's fine if it's in sycn with App Service. The problem is that in my experience it very rarely is.

We're making changes so for that to happen as part of the same deployment.

Microsoft.NET.Sdk.functions should lock this stuff so that you can't install the wrong versions too

It does that today. JSON.NET is one the the locked dependencies, but explicit references can (and well) override this behavior.

There are a bunch of other Azure dependencies that don't work right too that you have to be very careful to get the right versions.

Do you have more details on Azure dependencies you've had problems with? We'd definitely like to track (and address, or at least document) anything customers run into issues with

JohnGalt1717 commented 5 years ago

@fabiocav Excellent on the first one. Would still like to see a /depends on the func.exe.

Second item: Ok.

Third Item: KeyVault and RestClient I believe were the 2 that we noticed issues with.

gabrielbaptista commented 5 years ago

@fabiocav , we are getting this problem with EFCore 2.2.4 and NetCore 2.2. Rolling back EFCore to 2.2.1, the problem ends, but it is kind of annoying discover that.

hjpsievert commented 5 years ago

I have had similar problems as other folks in this thread, trying to get UserManager and SignInManager to work on top of the standard ApplicationDbContext that you get from the VS template when you use Microsoft.AspNetCore.Identity. I get the Type error with the DBContext that others have pointed out with version 2.2.4 of Microsoft.EntityFrameworkCore and Microsoft.EntityFrameworkCore.SqlServer. Those errors go away when both are downgraded to 2.2.3.

Microsoft.AspNetCore.Identity and Microsoft.AspNetCore.Identity.EntityFrameworkCore are both on 2.2.0 which is still the newest version, Microsoft.NETCore.App is 2.1, have not tried this with 2.2.5. With the downgrade I am able to access all aspects of user data and functions that I am trying to migrate from a standard MVC Web Application into Azure Functions.

My startup class is below:

  public class Startup : FunctionsStartup
  {
    public override void Configure(IFunctionsHostBuilder builder)
    {
      builder.Services.AddDbContext<ApplicationDbContext>(options => {
        options.UseSqlServer(System.Environment.GetEnvironmentVariable("identity_connection"));
      });
      builder.Services.AddIdentityCore<ApplicationUser>()
        .AddSignInManager()
        .AddEntityFrameworkStores<ApplicationDbContext>();
      builder.Services.Configure<IdentityOptions>(options =>
      {
        options.SignIn.RequireConfirmedEmail = true;
        options.Password.RequiredLength = 8;
        options.Lockout.DefaultLockoutTimeSpan = TimeSpan.FromMinutes(10);
        options.Lockout.MaxFailedAccessAttempts = 10;
      });
    }
  }
espray commented 5 years ago

@hjpsievert - I have not gotten identity to inject correctly. Does SignInManager<ApplicationUser> inject correctly into your Function?

hjpsievert commented 5 years ago

@espray Initially I thought it did, it certainly gets injected and you can inspect it when on a break point. However, when I checked after your question, I realized that the Context is not set and that means it is really not usable like that. UserManager works just fine, by the way, and connects to the Identity Database without problems.

I am experimenting with Azure Functions to possibly migrate a MVC Web API which primarily provides the backend for a mobile app. I am not really using the web site generated, but did not want to reinvent login functionality (encryption, hashes, two-factor authentication, etc.) and used ASP Identity because of that. Users actually sign in inside the app and I access the login function through the API. I was thinking of eventually offering the app via web as well.

After your post, I did some experimenting to see whether I could get the SigInManager to work, but without success so far. I typically get an error about a missing authentication handler and am stuck at this point. I think the fact that there is not way in the Azure Functions Startupto invoke the app.useAuthentication call used in the MVC Web API that adds the authentication middleware, is the underlying reason. I tried to connect the SignInManager to my function's HTTPContext, but that did not work either.

Anything you have tried that sound promising?

espray commented 5 years ago

@hjpsievert doing the same thing. Running into the same problem. No solution yet. I have not tried Dynamic Authentication approach.

jeffhollan commented 5 years ago

Thanks all - seeing this in my repro's as well. If using Entity Framework Core today go with 2.2.3. 2.2.4 causes issues during service registration in startup. Not sure why, but flagging this as it's come in a lot over the last few weeks.

/cc @fabiocav

espray commented 5 years ago

@hjpsievert we should move the Auth/Auth Identity conversation to #4485

hjpsievert commented 5 years ago

@espray makes sense, I saw that thread when I was looking for information

jeffhollan commented 5 years ago

Moving to a P1 as the reason is the hosting platform today is running .NET Core 2.2.3, and the local tools use a self-contained build as well on 2.2.3 currently. So this won't be an issue once 2.2.4 rolls out in the underlying infrastructure. Am working to understand how best we can capture and document current supported versions

benm-eras commented 5 years ago

I initially raised an issue over this in August last year, and given the recent update to allow proper DI in Azure Functions I thought I would give it a whirl and I was disappointed to see that the exact same issue persists 10 months later! I have updated my demo repro the demonstrates an MVC project working fine, and an Azure Functions project breaking when trying to use the same IdentityDbContext.

Does anyone know if this will be resolved in ASP.NET Core 3?

fabiocav commented 5 years ago

@benm-eras I just commented on another issue (same topic) you were mentioned. The hosted version is on 2.2.5 now.

Updates landing for .NET Core 3.0 will have the same behavior. App Service does lag behind the releases for a few days, and we adopt them as soon as they're available as part of the platform.

JohnGalt1717 commented 5 years ago

@fabiocav If by "few days" you mean "weeks to months" sure.

But vitally this isn't an acceptable answer because these patches are often major security vulnerabilities so you're leaving all functions that are running on .NET Core open to hacking because you're not in sync with the release.

This should be synchronized within MS. There's no excuse. There might be if you were a 3rd party company, but you're not. This should all be part of the build process for new releases of .NET Core.

hjpsievert commented 5 years ago

I am with @JohnGalt1717, both from a perspective of security updates, but also because having to manually ensure version compatibility through trial and error and remembering to keep checking on it is not something I expect in products from Microsoft.

fabiocav commented 5 years ago

@JohnGalt1717 I'm being transparent about the current state of the platform. Today, those updates are not deployed to the underlying platform at the same time they are published. We're working to reduce the time to adoption to what I stated above. We'll continue to work on improving this, which includes improvements to communication and clear message from the runtime. I understand that this is not ideal, but I do have provide accurate information about how this works as it is something customers need to consider when adopting a managed platform.

From an App Services perspective, an update like that cannot be blasted to thousands of customers without proper validation and announcement. As an Azure service, we (App Services and Functions) follow strict deployment guidelines that, in addition to validation, include phased rollouts in different regions. Deploying those updates before customers have access to RTM bits for validation is problematic, as they would be hit with the new version in production before they have a chance to validate them.

If you want to be in full control and use the functions programming model, your option would be to build and deploy as a private site extension (using app service plan) or use one of our container hosting options.

@hjpsievert .NET Core updates will be posted in the announcements (https://github.com/Azure/app-service-announcements/issues).

We understand this should be improved, but again, I'm providing information on how things work today and how to avoid the issue that was reported here. I appreciate the patience as we work on improving this process.

JohnGalt1717 commented 5 years ago

@fabiocav I get it, but Azure Functions host should be part of the framework, it shouldn't be managed by the Azure team at this point. It's as fundamental as asp.net core. My point is that you're taking baby steps when the entire ownership of the project is in the wrong place and needs to be fixed so that it is part of the dotnet project and built as part of it. How the azure team then takes that code and rolls out should work exactly the same way that asp.net core is rolled out to App Services.

Because right now, we can't even TEST in VS.net with the latest .net core runtime until you guys get your acts together so we can't even do our testing while you're doing rollout. Nor can we do what you suggest and use docker (which is a huge PITA in Azure Functions and I'd argue barely usable) because we don't have the runtimes to do it because they're not released yet.

See what I'm saying?

If this was part of the dotnet project and built exactly like asp.net core was, and made available the same way including VS.net extension etc. and docker images, then yes, we could do what you suggest and do so immediately and the rest could just wait for it to be deployed to azure app services and dev ops. (and you could easly do what the asp.net core team does with their extension to app services)

Until that fundamental change happens, Azure functions is always in a position of a massive security risk AND a pain in the ass to work with with other code and dependencies making me want to never use Azure Functions ever again because it just isn't worth the pain and risk.

Someone from your team asked why people aren't using Azure Functions with .NET on Twitter. This was my answer. It got a lot of feedback. You've created DLL hell part 53423423 because you're approach is just wrong.

fabiocav commented 5 years ago

@JohnGalt1717 would making the host and Core Tools available in a preview channel help you with some of the issues? That would be the *SDK/Framework portion of Functions.

Even if the .NET Core team was managing this, they would not be able to update the hosted/managed service portion of the product (Functions on Azure) on day one because of the concerns I've outlined above.

JohnGalt1717 commented 5 years ago

Yes that would certainly help.

However, just as asp. Net core previews are easily added to azure app services using the extension they have, the same should be possible for functions.

And, all of this stuff needs to be released and at least available 0 day with net core releases, public and preview. Then what you do with azure is up to azure and hopefully you'll grace us with an extension for previews and to force 0 day public release without having to wait for standard staged rollout.

fabiocav commented 5 years ago

We'll continue to execute on the plans I mentioned for the initial mitigation steps, which will hopefully help alleviate the issue and improve communication, but exposing something like that, with an opt-in model, is something tangible and I'll work on the plans on our side. This will require some time as there's some coordination to be done across a few different teams, but once there's something more concrete to share, I'll open up the issues for tracking.

jeffhollan commented 4 years ago

@fabiocav this issue is pretty old, and I know at least for me I'm using the lastest 3.1 SDK and 3.1 Entity Framework with no dramas. Can we close this? At the very least what's the actino on our side so I can use the priority label. LMK

jeffhollan commented 4 years ago

Closing now. Here's a sample of it working now in the latest bits. If you still see any issues please create a new issue. Thanks

https://github.com/jeffhollan/functions-chsarp-entitycore-3