search

dotnet / standard

This repo is building the .NET Standard
3.06k stars 428 forks source link

.NET Standard issues on .NET Framework 4.7.1 #567

Closed AlexGhiondea closed 5 years ago

AlexGhiondea commented 7 years ago

Summary

Applications that target the .NET Framework 4.6.1 and use .NET Standard libraries may experience System.TypeLoadException exceptions when running on the .NET Framework 4.7.1

Symptoms and root cause

The .NET Framework 4.7.1 adds support for .NET Standard in-box. As part of this change, a few assemblies that used to ship independently from the .NET Framework have now been added to the .NET Framework. As these .NET Standard assemblies were not part of the .NET Framework prior to .NET 4.7.1, they were deployed with the application, and a binding redirect was added to the application's app.config file.

When running on the .NET Framework 4.7.1, where the assemblies are now part of the Framework, the application now has two types with different identities, and that leads to a TypeLoadException. One type comes from the app-local assembly, and the other one comes from the .NET Framework assembly.

Also reported here: https://github.com/Microsoft/dotnet-framework-early-access/issues/9

The following types are potentially impacted by this issue.

Plan

We have identified the problem and we will ship a fix for .NET Framework 4.7.1 in the next few months and in the next version of .NET Framework.

Workarounds

There are two ways to work around this issue:

leastprivilege commented 7 years ago

Re-target your application to target the .NET Framwork 4.7 or .NET Framework 4.7.1.

What does that mean for library authors?

leastprivilege commented 7 years ago

OK - since no-one cares to answer - maybe I need to elaborate.

We have reports that applications using our libraries have the problem described here. See links above.

Can we as a library author do anything to prevent these problems - or is that the problem of the application? Shall we e.g. at least for the time being add a net471 target that uses the framework references instead of nugets?

Or will you just fix that issue (hopefully not in a couple of months - but earlier) and we'll just forget about 4.71...?

sharwell commented 7 years ago

@leastprivilege I expect you will get an answer sooner rather than later. I'm trying to learn more about exactly what steps a user takes to enter this scenario, since it seems to affect some users with .NET 4.7.1 but not others.

AlexGhiondea commented 7 years ago

@leastprivilege, @sharwell I understand your concerns. Here is more information about what is happening.

The scenario that is broken is the following: An application that targets .NET Framework 4.6.1 and references a .NET Standard 2.0 library. In addition to that, the application needs to also directly reference System.ValueTuple library (or any of the assemblies that contain the types listed above) and have a binding redirect for that version of the library. In addition to that, the application needs to exchange the System.ValueTuple type (or of one of the types above) with the library. There are many ways this can happen (pass an instance of a type between the two, implement an interface that uses System.ValueTuple in the signature, etc).

When running on .NET Framework 4.6.1 the application will use the System.ValueTuple assembly that ships with the application (since System.ValueTuple is not part of the .NET Framework). The library also uses the System.ValueTuple type that ships with the application because the netstandard.dll that ships with the application will type-forward to the System.ValueTuple contract.

When running on .NET Framework 4.7.1. the application will use the System.ValueTuple assembly that ships with the application. This is because the application has a binding redirect to the explicit version of System.ValueTuple that comes with the application. The library will end up using the System.ValueTuple type that comes from .NET Framework 4.7.1. The reason for this is that the library will load the netstandard.dll from the GAC (since now netstandard.dll is part of the framework) which type-forwards directly to the implementation in .NET Framework 4.7.1.

When the 2 (application and library) exchange an instance of System.ValueTuple you will end up with 2 different identities for a single type (i.e the types don't unify) which causes an exception.

We are planning to fix this by changing the way netstandard.dll's type-forwards are generated. Instead of directly going to the implementation, they are going to type-forward to the contracts for the types that used to ship in separate packages and are now in the .NET Framework.

@leastprivilege adding a net461 asset is going to be the fastest way for you to unblock your customers. If you also have a netstandard 2.0 asset then you will ensure that the code remains compatible with .NET Standard and that it can run on 4.6.1

leastprivilege commented 7 years ago

Thanks for the clarification - does this look right to you:

https://www.nuget.org/packages/IdentityModel/2.15.0-preview1

Tornhoof commented 7 years ago

May I suggest a consolidated post on all the issues currently with .NET Standard and the relevant netfx frameworks? I see that you tag your posts as announcement, but a consolidated view would be nice and put it somewhere visible, e.g. the dotnet blog or even that shiny purple dot.net website.

Kieranties commented 7 years ago

@AlexGhiondea I assume this issue would also be apparent for applications targeting net462 and consuming netstandard2.0 libs in a net471 environment?

sharwell commented 7 years ago

@AlexGhiondea I have an application which, as far as I can tell, falls exactly into the set of impacted applications. For the majority of users, the scenario is not causing problems. However, in a small subset of cases the users are experiencing problems.

The case we are hitting involves IncrementalHash:

System.Security.Cryptography.Algorithms System.Core Result
4.7.2556.0 4.7.2556.0 Works
4.7.2556.0 4.7.2102.0 Fails

We don't know how a user can enter the latter case, where one library is updated but the other is not.

joperezr commented 7 years ago

@sharwell yes, System.Security.Cryptography.IncrementalHash also belongs to the same bucket than System.ValueTuple (implementation used to live in a nuget package, but is now inbox on netfx) so most likely you are hitting the same exact issue and will be fixed with the proposed fix by @AlexGhiondea of building the netstandard facade such that it types forwards to contracts first when possible.

sharwell commented 7 years ago

@joperezr The strange thing to me is I'm not hitting the described problem. The original description and follow-up describe a scenario in which this problem will occur. However, the description is current incomplete as many users who meet these conditions do not experience the described problem.

AlexGhiondea commented 7 years ago

@sharwell I would like to know more about the environment that is happening in.

Because of the different versions of System.Core you might be in a strage case where the facade (System.Security,Cryptography) has a type-forward to the IncrementalHash type but that type is not in the System.Core.dll you have.

Do you have a repro of this case?

AlexGhiondea commented 7 years ago

@leastprivilege the package you pointed at looks ok to me.

tstojecki commented 7 years ago

@AlexGhiondea can you give us an idea for what else might be missing? I just tried @leastprivilege package (2.15.0) which has been compiled with that extra net461 target and the dependent packages - see the screenshot I posted here https://github.com/IdentityModel/IdentityModel2/issues/70

We're still ending up with MissingMethodException System.Tuple when hitting the controllers. It is a aspnet core 2 web app targeting net461.

leastprivilege commented 7 years ago

@AlexGhiondea could you please work with @tstojecki and us to resolve that issue. We are relying on your technology - and I'd love to see a bit more passion on your side fixing problems like this.

AlexGhiondea commented 7 years ago

@leastprivilege sorry for the delay in replying. @terrajobst - any ideas?

@tstojecki @leastprivilege can you please share a repro so that we can investigate?

joperezr commented 7 years ago

I'm inspecting the net461 binary from your package @leastprivilege and something did catch my eye. I saw that you have a reference to System.Net.Http with a higher version that the one for the .NET 4.6.1 Targeting pack (You are referencing version 4.1.1.2 when the one on the targeting pack is 4.0.0.0) This leads me to believe that when you are building this IdentityModel.dll you have other .NET Core based dependencies which is causing your library to build against the Support package who is bringing these higher versions. If this is the case, then your consumers need to pull in that Support package or else they can end up being in torn states like @tstojecki . Do know that this specific issue with the higher versions is one that we are currently working to fix in the next release of VS, but for now, @tstojecki can you try adding to your net461 aspnet core project the following code:

<PropertyGroup>
   <_HasReferenceToSystemRuntime>true</_HasReferenceToSystemRuntime>
</PropertyGroup>

Let us know if that fixes your issue.

leastprivilege commented 7 years ago

This leads me to believe that when you are building this IdentityModel.dll you have other .NET Core based dependencies which is causing your library to build against the Support package who is bringing these higher versions.

Sorry I don't understand what that means.

Here's the repo

https://github.com/IdentityModel/IdentityModel2

Please have a look and tell me what is wrong.

Are there any docs I am missing that could help people build libraries that actually work?

joperezr commented 7 years ago

Please have a look and tell me what is wrong.

Nothing is wrong with your library, the problem here is the tooling. Sorry for not making that clear 😄.

Sorry I don't understand what that means.

What I meant by that is that one of your dependencies (when building your library for 461) is either netstandard, or System.Runtime based. This causes our logic to include what we call the support package as references when building your library for 461. This support package, contains shims that will enable those netstandard/System.Runtime references to work on 461. The problem, is that those shims have higher assembly versions than the ones that we have inbox (this is a bug in our side) which causes your resulting IdentityModel.dll to depend on System.Net.Http version 4.1.1.2 instead on the one from 4.0.0.0 which comes inbox. These issues are being addressed for the next version of the VS tooling (15.5) but unfortunately on the mean time, consumers of your library would have to add a property to their project in order for things to work. That property is the one I sugested to @tstojecki above.

leastprivilege commented 7 years ago

ok thanks for the clarification.

erikbra commented 7 years ago

State of .NET Tooling for the time being... not excellent. At all.

tstojecki commented 7 years ago

Thanks for your input @joperezr and @leastprivilege As that was a blocking issue for us we ended up upgrading and targeting 4.7 in our projects, which resolved it. I have just tried switching the projects back to 4.6.1 but I no longer get that error. Not sure, could be that I would have to uninstall .net framework 4.7 and I am not in a great place to do that. If I get back to where this issue occurs again, I will be sure to try the config option.

Shashanka77 commented 7 years ago

@joperezr and @AlexGhiondea I have two issues I would like to discuss.

  1. "What I meant by that is that one of your dependencies (when building your library for 461) is either netstandard, or System.Runtime based" - In the above sentence, I did not understand the phrase "either netstandard or System.Runtime based" what does this exactly mean?

  2. We have a WPF application targetting .NET Framwork 4.7.1 that consumes several .netstandard2.0 libraries. The application runs fine in Dev ENv (ie. VS 2017 debug mode). But after installing the WPF app via publish option, We are getting runtime error at application launch "Could not load file or System.Net.Http, Version 4.1.1.2 or one of its dependencies. The System.IO.FileNotFoundException could not load file or assembly System.Net.Http, Version 4.0.0.0 or one of its dependencies.

Do recommend to resolve this by compiling one of our several .netstandard lib with following?

<_HasReferenceToSystemRuntime>true

I am not sure what assembly is actually trying to these two version of System.Net.Http and why both versions are not getting picked at runtime.

Thank you

joperezr commented 7 years ago

In the above sentence, I did not understand the phrase "either netstandard or System.Runtime based" what does this exactly mean?

When we say that an assembly is netstandard based is one where the System.Object type it was built against was in netstandard.dll. A System.Runtime based, it's the same but when System.Object lives in System.Runtime.dll. Usually, if your assembly targets netstandard1.* then you are probably System.Runtime based, and if you target netstandard2.0 then you are netstandard based.

For the second issue, what version of VS are you using in order to build your app? We have a few known issues with System.Net.Http when trying to run a 4.7.1 app with a netstandard2.0 component. These issues are already fixed on the next version of VS (15.5), So I do expect that once you upgrade to 15.5 then these issues would go away.

Do recommend to resolve this by compiling one of our several .netstandard lib with following?

For the meantime unfortunately, setting the property won't help you when you are targeting 4.7.1, as it would still not fix your issue. Is there a specific reason why you need to target 4.7.1 with your WPF app? I ask this because one workaround, could be to retarget your WPF app to a lower version of the framework (4.6.1 - 4.7) and set the _HasReferenceToSystemRuntime property on the WPF project in order to get things working while we ship 15.5.

pranavkm commented 7 years ago

@joperezr is there a build of the 2.2. preview SDK that addresses this?

joperezr commented 7 years ago

this specific issue is fixed on some targets files that are not really part of the sdk itself, but in a msbuild extension instead. 15.5 will contain these fixes.

Shashanka77 commented 7 years ago

@joperezr so my understanding that .net 4.7.1 msbuild version not able to publish wpf app correctly is right?

AlexGhiondea commented 7 years ago

@Shashanka77 i am not sure what you are asking. What version of VS are you using?

joperezr commented 6 years ago

He is talking about the publish to Azure issue that I talked to you about. @Shashanka77 I was able to repro and we are currently investigating more deeply what is the issue. I’ll update the thread when I know more info.

Shashanka77 commented 6 years ago

@joperezr Let me be more clear, detailed and specific. Actually after your suggestion, I have changed target framework of my WPF app to 4.7. Note that this app is still consuming several .netStandard 2.0 nuget packages. The application just runs fine locally (running exe from bin/debug). After I publishing the app locally, when I run the setup.exe from published location the installation fails with following exception. "Unable to install or run the application. The application requires the assembly System.Xml.Xpath.XDocument Version 4.1.0.0 be installed in the GAC first".

So I am assuming that this is because of consuming netstandard 2.0 packages. Is this assumption true? It seems that the application is looking for a particular version of dll (that is referenced by one of the consumed .NETStandard 2.0) and that is not present in the GAC and also published app folder also does not have this dll. Please note that I am publishing from VS 2017 Version 15.4.4

joperezr commented 6 years ago

@Shashanka77 Ok I think I finally understand what is going on in your case. I believe the issue you are hitting is #529, which is a bug where ClickOnce doesn't include the right files when installing your app. Specifically, it fails to understand that some assembly versions that you reference are higher than the ones that are inbox. This is causing you to fail to publish. We have contacted the ClickOnce team to try to get a fix to this soon, and also to try to get you a temp workaround in the meantime.

bjcull commented 6 years ago

I think Azure may have deployed something last night because I'm suddenly getting this problem in production (Azure App Service). Errors started a few minutes after 20 Dec 2017 00:00 UTC.

clairernovotny commented 6 years ago

@bjcull what region? App Services has been rolling out Server 2016 with .NET 4.7.1 on it over the course of the month.

bjcull commented 6 years ago

@onovotny Australia East. Makes sense that it would be towards the end of the month :)

jdasilva commented 6 years ago

Is there a way to track the release of the fix to the framework mentioned here? or is this thread the best way?

joperezr commented 6 years ago

This is probably the best place since we will communicate through here once the fix is rolled out to the public.

scottsauber commented 6 years ago

It would appear that fix is getting rolled out in WSUS now.

https://blogs.msdn.microsoft.com/dotnet/2018/01/09/net-framework-4-7-1-is-available-on-windows-update-wsus-and-mu-catalog/ judging by the 1st mention in the BCL section

joperezr commented 6 years ago

Yes, as @scottsauber said most of the problems covered by this issue are now available through Windows Update. Please go ahead and update your machines and let us know if the fix is working for everybody hitting this before.

mattbrooks2010 commented 6 years ago

@joperezr Do you know of any announcements / articles that indicate when the fix will be applied to Azure App Services?

joperezr commented 6 years ago

@mattbrooks2010 I'm not sure about Azure specifically, but in general, all new installs of 4.7.1 should automatically get the fix as well, which in Azure case will most likely mean that once they start supporting 4.7.1 then they will have the fix.

If the machine already had 4.7.1 on them, then they would have to manually install https://support.microsoft.com/help/4054856 to get the fix, but as I said, I believe that Azure doesn't support 4.7.1 yet, so once they do they should get this automatically.

mattbrooks2010 commented 6 years ago

@joperezr Thanks for the info. I think Azure must be running 4.7.1 already, as this precise issue has impacted us in production there out of the blue quite recently. Unless I’ve misunderstood something?

joperezr commented 6 years ago

AFAIK 4.7.1 isn't installed on Azure machines by default, I believe that today if you need to have 4.7.1 on an Azure machine you are required to install it as part of your deployment, but I might be wrong about that. Are you sure that this is the root cause of the issue that you are seeing? If so, then I would contact Azure to see if there is something you need to do in order to get this patch installed.

mattbrooks2010 commented 6 years ago

Thanks again for the info.

Yes, I believe this is the root cause of the problem I’m seeing. The affected app was functioning fine for weeks until it consistently started failing due to an unexpected runtime issue on startup. There were no code changes and as far as I know no deployments that triggered the issue. The error thrown led me here — removing the assembly binding for ValueTuple fixes the issue. Tomorrow I will try the other workaround of re-targeting the app to net47.

mattbrooks2010 commented 6 years ago

Re-targeting the app to net47 has mitigated my issue.

Latency commented 6 years ago

Image.Load() => {"Could not load file or assembly 'System.Runtime, Version=4.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.":"System.Runtime, Version=4.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a"}

I am just trying to run it as native 4.7.1 without CORE for Main UI.. I have my dependancy libs x-targeted for both Core2.0 and 4.7.1 but that shouldnt do anything. However, the MSBuild configurations that support this WILL make a difference.

I am positive the problem is with the build & release of this package... and I will prove it!

Because I didn't have any problems prior to installation the issue is I do not need a binding redirect from the package from System.Runtime. It is now integrated as part of STD.

System.Runtime from NuGet package (latest) installation targets 4.6.2. It is incompatible and will not load if copied nor will it support any targeting assemblies that are build with framework versions > 4.6.2. Meaning, assemblies which use .NET 4.7.x will NOT load!

In fact, if you browse to include it as a referenced assembly, you get a yellow exclamation box. More than likely, conflicting or incompatible.

I am thinking the only thing I know how to do is revert to stable package 1.5 but thats 6 mo old and the constructs are different for things.

I will try to download the src and repack it today against v4.7.1 but I would also like to mention: System.IO.Compression should be added to the list.

'System.IO.Compression', Version=4.2.0.0 != Version=4.0.0.0

Moreover, adding versioning to 'App.config; will not resolve this. Similarly, \true\

Partial Workaround

I was able to include this library from Reference assemblies and then under Properties, change the alias to "newVer" or whatever you wish.

Then in your project in the source code, you need to add: extern alias newVer;

Where you then pre-pend the namespace like so.

using (var zip = ZipFile.Open(backupFilePath, newVer::System.IO.Compression.ZipArchiveMode.Create)) { ... }

This will not work for 'System.Runtime' since there is not one to be found in reference assemblies anymore.

Release / Fix / Repack

SixLabors.ImageSharp.1.0.0-beta3.nupkg - by Latency

Screenshot

image

See referenced link below for release notes and more information.

antonfirsov commented 6 years ago

Guys, do you think @Latency's ImageSharp issue is related to this?

We have a .NET Standard 2.0 target in our NuGet package that is being used by a .NET Framework 4.7.1 project.

I was not able to reproduce it. In an empty project everything just works fine.

tomasaschan commented 6 years ago

This needs to be listed and linked in the release notes for .NET 4.7.1. After Windows installed updates on Friday of last week, I've spent days looking for a fix or even a workaround for this before finally finding at least the latter here. (Luckily I happened to control all the libraries I had this problem with, so I could easily add a net461 tfm to the target frameworks in all cases.)

nzain commented 6 years ago

comment deleted. (I hit the wrong ticket with my comment, sorry)

EvanHaklar commented 6 years ago

@joperezr Are there any updates on the ClickOnce issue described by @Shashanka77 above? I'm seeing precisely the same behaviour on VS 2017 15.6.2 for a 4.6.1 (test) console app which references both 4.6.1 libraries and .net standard 2.0 libraries - with the same error message verbatim.

joperezr commented 6 years ago

Not yet unfortunately, last time I asked the ClickOnce team they said that the fix was on track for making it to 15.8

gulbanana commented 6 years ago

that’s a long time to have the once-flagship install system not functioning. i’m sure windows store is recommended for new projects, but there are a great many older business apps out there which are now broken and can’t be updated. my organisation had to switch two projects from clickonce to squirrel, an unanticipated burden.