Binding redirect support

[mamaso]

Per this customer issue:

http://stackoverflow.com/questions/40816285/azure-functions-with-nuget-packages-that-have-different-versions-of-the-same-dep#

[MisinformedDNA]

@RudyCo Sure, but it's not hard and it's a reasonable workaround. VS2017 Update 3 (still in preview) might have better tooling, haven't checked.

@jnevins-gcm Precompiled C# allows you to set redirects, 99% of which are set automatically via NuGet. I'm using precompiled C# and I haven't found any similar issues. What are you seeing?

[jnevins-gcm]

I am pretty sure using precompiled c# code doesn't actually use whatever binding redirects you have in your app.config...

[jaredcnance]

@MisinformedDNA for a concrete example (using a "precompiled function"), try referencing the latest version of Autofac and Autofac.Extensions.DependencyInjection. Then call the ContainerBuilder.Populate(...) method. It will compile fine but you will get a runtime MethodNotFoundException. Things like this are forcing me to add extension method shims. Another example can be found with log4net and really any other assembly that doesn't match the versions referenced by the runtime, all of which will result in runtime errors.

[MisinformedDNA]

Ignore everything I said. My packages aren't as up to date as I remembered. Carry on.

[fabiocav]

@MisinformedDNA pre-compiled functions do not have redirect support. The load behavior there is no different than references using the shared model (loaded in the load-from context)

A private assembly deployment is a current workaround for a redirect where you have a binding failure, but a bit cumbersome.

@jnevins-gcm yes, the goal is to have that done in the short(er)-term to unblock some of these scenarios.

[ed-ilyin]

@anyone, could you please, point me at a code example to implement a workaround? I'm getting errors like one below:

System.MissingMethodException: 'Method not found: 'Microsoft.FSharp.Core.FSharpFunc`2<System.String,Microsoft.FSharp.Core.FSharpChoice`2<Json,System.String>> Json.get_tryParse()'.'

my assembly resolve handler does not catch anything useful.

[aarondcoleman]

I hate to pile on, but just as a data point our migration to Functions is pretty much on hold until this can be resolved. We have way too many dependencies to juggle with nuget packages that all want higher versions of libraries, like WindowsAzure.Storage 8.x and being shackled to older versions is a showstopper. I have to agree with some comments above that 6+ months out for a solution relegates Azure Functions to more of a one-off tool for doing simple self-contained operation work, as opposed to what we were hoping for, which was to move parts of our app off of Service Fabric but still leverage out internal core shared library. Hoping this helps prioritize this as I think Functions, especially with the new VS tooling and precompiled workflow, are a sorely needed tool in our toolbox.

[solvingj]

This is an area where the .NET mechanism for loading assemblies is simply not designed to support this level of flexibility and control. It's extremely complicated. I worked extensively with Java classloaders on the JVM, and while they have their own issues and are getting an overhaul, they were capable of dealing with this situation.

I believe the situation warrants completely new functionality in .NET, and the engagement of the .NET core teams to discuss it and look at other languages to see what prior art exists. In short, to solve this problem effectively for Functions, or any other future "run code on demand" concepts, developers need the ability to create fully isolated and controlled mini environments. Anything based on App Domains isn't going to cut it.

[Gargsurbhi]

I followed @npiasecki post to add custom handler to resolve redirect binding issue. It is working locally but not working after publishing to Azure. I am getting following exception.

System.MissingMethodException : Method not found: 'Microsoft.Azure.Documents.Client.TransientFaultHandling.IReliableReadWriteDocumentClient Microsoft.Azure.Documents.Client.TransientFaultHandling.DocumentClientExtensions.AsReliableSystem.MissingMethodException : Method not found: 'Microsoft.Azure.Documents.Client.TransientFaultHandling.IReliableReadWriteDocumentClient Microsoft.Azure.Documents.Client.TransientFaultHandling.DocumentClientExtensions.AsReliable(Microsoft.Azure.Documents.Client.DocumentClient, Microsoft.Practices.EnterpriseLibrary.TransientFaultHandling.RetryStrategy)'.

[theclifmeister]

So... any progress on this?

[mmaitre314]

Hit the failure while trying to use ADAL auth in Azure Functions:

Microsoft.Azure.Services.AppAuthentication: Could not load fileor assembly 'Microsoft.IdentityModel.Clients.ActiveDirectory.Platform, Version=3.14.2.11, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified.

The lack of support for app.config is confusing.

[MisinformedDNA]

Is there a page with all the internally referenced DLLs and their runtime versions. For instance:

WindowsAzure.Storage: 7.2.1 Newtonsoft.Json: 9.0.1

[paulbatum]

I would recommend you look at the binding redirects in our web.config: https://github.com/Azure/azure-webjobs-sdk-script/blob/dev/src/WebJobs.Script.WebHost/Web.config

However be aware that at some point soon the dev branch will switch over to tracking our .net core based 2.0 release. At that point there will be a v1.x branch and that is the branch you'll want to check for v1 function apps.

[TimPosey2]

Is there any way to just get the easy stuff upgraded? I've tried several workarounds on assembly binding redirects to no avail.

Problem for me is I need Microsoft.Azure.Documents.Client to be v1.17.0 (currently at 1.13.0) in order to better handle JSON Serialization for CosmosDB. Should be a benign thing.

[BowserKingKoopa]

I'm running into this issue all the time. Lately, Azure Functions QueueTrigger is unable to deserialize CloudQueueMessage because it's using a different Azure Storage version than I am. I've had to move a lot of Azure Functions back to WebJobs and if there isn't a fix for this soon I might have to do a full on retreat. This is looking like Azure Function's tombstone to me.

[shanfangshuiyuan]

Is there any progress? I am trying to use lastest Kusto client nuget package, which requires Newtonsoft.Json version > 10.0.3, but Microsoft.NET.sdk.Functions explicitly asking for Newtonsoft.Json version to be 9.0.1. Any workaround?

[wimagee]

Would really appreciate an update from the team on this. It's a year now since this issue was raised and there's been nothing since July on even the short-term partial fix which has been bumped along each month #1716 . At that stage it was 6+ months to a full fix - my fear is that we're still 6+ months from a proper solution. If that's the case so be it, but we need some clarity as at the moment we're treading water in the hope that this comes through. We're at the point of deciding that it isn't going to happen even in the medium term and looking to other options for this kind of workload.

Added to the binding issues with .NET Standard 2 libraries (EF Core 2.0 etc) in net462 projects and all the csproj changes we're in a world of pain with .NET on Azure at the moment unfortunately.

[tomkerkhove]

Adding @christopheranderson & @lindydonna, can you elaborate on this please or include the relevant people please?

[fabiocav]

Apologies for the lack of updates here.

Issue #1716 is the targeted fix for some of the scenarios impacting the current runtime version. As pointed out by @wimagee, the issue has unfortunately been bumped due to other competing work, but I am targeting this sprint (or next, at the latest) for completion.

It's worth noting that, as stated on the issue, that addresses some common issues with redirects that would typically require custom code, but not all (the feature will essentially allow for binding redirects that change how probing and loading happen in the function assembly context/metadata resolver).

@wimagee if you're referring to binding issues .NET Standard and functions (and not .NET Standard 2.0 in general); we have updates being deployed with the next release due to happen within the next week to address problems with that.

This is also a high priority item for the 2.0 runtime, and while there hasn't been an update here, work is happening there to ensure this issue is addressed.

[forki]

so is it correct that you don't support binding redirects and you don't expose something like a palet.lock file with all the deps versions that you have? How can this possibly work?

[Szer]

Good news everyone!

Today I was able to publish pre-compiled function targeting netstandard2.0 using all packages I had problems in past such as: FSharp.Data 2.4.3 Newtonsoft.Json 10.0.3 Microsoft.Azure.Management.DataLake.Store 2.2.1 WindowsAzure.Storage 8.6.0

and... It worked without trouble!

You have to switch to beta runtime 2.0.11415.0 (beta) in Function App Settings though. But it works and it is great.

[forki]

Good for you ;-) But doesn't solve underlying issue that we can pin our deps versions

Am 06.12.2017 16:59 schrieb "Szer" notifications@github.com:

Good news everyone!

Today I was able to publish pre-compiled function targeting netstandard2.0 using all packages I had problems in past such as: FSharp.Data 2.4.3 Newtonsoft.Json 10.0.3 Microsoft.Azure.Management.DataLake.Store 2.2.1 WindowsAzure.Storage 8.6.0

and... It worked without trouble!

You have to switch to beta runtime 2.0.11415.0 (beta) in Function App Settings though. But it works and it is great.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Azure/azure-webjobs-sdk-script/issues/992#issuecomment-349683694, or mute the thread https://github.com/notifications/unsubscribe-auth/AADgNFQ1u47ACn5t7HMB3cXaUN2wxh_fks5s9rnQgaJpZM4K9N-C .

[shyamal890]

@fabiocav @forki Can you please update us on when a fix would be available? Can't an alpha be released on nuget updating the dependencies to latest versions?

[paulbatum]

@shyamal890 There are prerelease nugets already available for Azure Functions 2.x / Webjobs 3.x with updated dependencies:

https://www.nuget.org/packages/Microsoft.Azure.WebJobs/3.0.0-beta4

If you create a brand new function app in Visual Studio and select the (v2) option it will automatically reference these.

[forki]

That is sooo dangerous without redirects...

Am 18.01.2018 19:04 schrieb "Paul Batum" notifications@github.com:

@shyamal890 https://github.com/shyamal890 There are prerelease nugets already available for Azure Functions 2.x / Webjobs 3.x with updated dependencies:

https://www.nuget.org/packages/Microsoft.Azure.WebJobs/3.0.0-beta4

If you create a brand new function app in Visual Studio and select the (v2) option it will automatically reference these.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Azure/azure-webjobs-sdk-script/issues/992#issuecomment-358731206, or mute the thread https://github.com/notifications/unsubscribe-auth/AADgNEeqLOFsgQkXVnPrxN6mD3or561gks5tL4fEgaJpZM4K9N-C .

[paulbatum]

@forki Not sure what you mean? These are prerelease packages for the next major version.

[forki]

I mean that it's hard for your users to guess against which versions they need to compile.

Am 18.01.2018 19:45 schrieb "Paul Batum" notifications@github.com:

@forki https://github.com/forki Not sure what you mean? These are prerelease packages for then next major version.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Azure/azure-webjobs-sdk-script/issues/992#issuecomment-358742795, or mute the thread https://github.com/notifications/unsubscribe-auth/AADgNLCBz1Q2rxttc3yco6_KwFSLs8Coks5tL5EvgaJpZM4K9N-C .

[ciarancolgan]

@paulbatum Hi Paul, I'm not sure what you mean by the line: "If you create a brand new function app in Visual Studio and select the (v2) option it will automatically reference these." If i create a new Azure Function project in VS2017 v15.5.4, it creates it as a .NET Framework project, which has the old, lower dependencies. Where's the "v2" option you mention? Thanks! Ciaran

[MisinformedDNA]

@ciarancolgan This what I see when I create a new Function project.

[paulbatum]

Make sure you update to the latest version of the tools:

[ciarancolgan]

@paulbatum the Tools upgrade did the trick, thanks!

[shyamal890]

The pertinent question to ask is what are the breaking changes in V2 and does it support .NET 4.6 sub assemblies?

[shanfangshuiyuan]

@paulbatum How to upgrade an existing function1.0.7 to 2.0 in visual studio?

[MisinformedDNA]

@shanfangshuiyuan Function SDK 1.0.7 is the latest and targets .NET Standard 2.0.

[MV10]

Today's Unreasonable Json.Net Challenge: Upgrade Functions SDK 1.0.7 to 1.0.8.

It's a patch release, How Hard Can It Be?™

1. Remove json 10.0.3 that the rest of the world is using
2. Remove five library packages that depend on 10.0.3 (or newer...)
3. Revert to json 9.0.1
4. Now 1.0.8 refuses to install, it declares another version-specific dependency on WebJobs 2.1.0
5. Remove WebJobs.ServiceBus 2.1.0-beta4 (because 1.0.7 declared that as a specific dependency...)
6. Bump the SDK to 1.0.8
7. Re-upgrade json back to 10.0.3
8. Re-add six packages
9. Realize I need to switch to Framework 4.7.1
10. Realize SDK 1.0.8 breaks my dependency injection library by assuming All Names Are Global
11. Push to repo, consider ditching Function Apps altogether
12. Start day-drinking

[paulbatum]

@MV10 I'm not sure how this moves the conversation forward. Of the problems that you listed here, those that relate to assembly loading are not new. Other problems you mentioned (such as something about assuming names are global) would be better filed as other issues so that they can be investigated. Similarly, if we screwed up our semver in the 1.0.8 release that should be discussed in another issue.

[BowserKingKoopa]

@MV10 @paulbatum I'm not sure any of us know how to move the conversation forward anymore. I've abandoned Functions completely, after investing a lot of time and effort into them, and I get irrationally angry when anyone at Microsoft mentions them. That's been my solution to the problem.

[aarondcoleman]

@paulbatum As someone also really affected by this, I'll just say that my experience has been that people tend to complain and become loud on things they really love and want to see improve. That's the case here for Functions.

My company spends $ six-figures on Azure yearly. Last year we went to BUILD (2017) and I went to every Functions talk, spent literally hours on the exhibit floor with the functions team strategizing, and we left sold that we could / should move our workflows to the new compiled functions. We got home, invested over a month of dev time, and then finally realized -- "oh man, we don't have any way to control the dependencies??" I really felt misled to be honest. To be told something is RTM, and then be told over and over we can bring our internal libraries over wasn't accurate. It was huge investment of time, a diversion of other priorities, but we had all the confidence. IMHO Functions marketing right now should have a huge asterisk on it so people don't have to find this thread to learn this limitation.

We've been told here, and in private emails to the team, that a solution was in the works, and we almost did what @MV10 did, which was plow ahead and just accept we'd have to live within the dependencies of Functions for the time being. But very quickly, other nuget includes start to conflict. We had to abandon Function all together and go back to Webjobs and Service Fabric.

We could argue over the tact of that post, but I'd say it's important for folks who are really stuck on this to speak up to help you and your team get "the higher ups" to give you more resources to make this a top priority. We've been told in the past that it is, but then not, as other things are more important and this keeps getting kicked down the road.

I just signed my team up for BUILD 2018 an hour ago -- I'm really hopeful that this year we'll be able make the switch. It'll be a huge cost savings for us, not to mention a simplification away from the frameworks around Service Fabric. No disrespect. ❤️to Functions -- we just want to use them.

[jaredcnance]

I am generally not one to jump on the complain train, but this is a serious issue. We are currently shipping pre-compiled functions to production and everytime we alter underlying dependencies, possibly unrelated to the functions themselves, we risk breaking our functions at runtime. I could live with this, if we had a repeatable and automated way to validate things had not been broken in development, but no such tooling exists yet and there seems to be 0 momentum in that direction (and also no support from MS to help the community develop it). I also am strongly discouraged from building any more services on Azure functions because with each new service, I am increasing my risk proportional the number of services I deploy.

We MUST be able to specify our own dependencies even at the expense of performance. This is possible today running dotnet core on AWS lambda.

[scottrudy]

Any chance the dependent libraries for Azure Functions with hard version dependencies could just be forked and refactored to use new namespaces?

[BowserKingKoopa]

Yes this. If Functions has to be its own walled off universe, maybe don't take a dependency on the json serialization library everyone else uses.

[fabiocav]

@scottrudy @BowserKingKoopa the problem here is not just internal use of serialization (and other) libraries in the runtime, but binding and type exchanges. Let's take storage, for example; you wouldn't be able to bind to a CloudBlockBlob from the SDK and use that as expected (which includes passing instances to other libraries and code you do not own) if the type was coming from a different assembly. Same applies to JSON.NET.

[fabiocav]

@jaredcnance, I know this is a bit different than what you were asking for on the issue you've linked to, but have you explored using deployment slots for validation/staging? That's one of the options you'd have. If that doesn't work, can we move to another issue to discuss the challenges?

[jaredcnance]

@fabiocav a bit different? They are one in the same. The entire purpose of testing is validating I haven't broken anything as I make changes. Now usually tests are intended for validating application logic, but I don't see why they can't also help us out with this issue until a solution is provided. Assuming it would be feasible to run tests on the same runtime as the functions themselves. Also, as I stated in that issue:

runtime binding errors should reproduce in tests. This probably means the test project should be executed by the function runtime

And yes, we are using staging slots today, but I—and I think everyone else here—would prefer to catch issues before we deploy. Suggesting that we deploy and just see if it works is not a solution.

[wimagee]

We made a decision last October that this wasn't looking likely to get resolved anytime soon despite assurances that bits would be in the next sprint etc. We went with other options which, while disappointing, was clearly the right decision.

From my perspective I cannot think what could possibly be more critical than this issue and the lack of progress in nearly 18 months leads me to believe that my (and many others!) understanding and expectation of the product has been wrong. From a technical point of view I suspect Functions was always intended just to run simple single function code; data manipulation/cleansing, service integration etc and that our desired use-case of shifting in existing and complex .NET workload is not a priority, and may in fact never be possible under the current implementation. If this is indeed the situation then someone needs to say so and avoid further frustration for those of us trying to beat a round peg into a square hole. Even if it's a very desirable square hole! :)

[fabiocav]

@jaredcnance by being a bit different, I was referring to my recommendation, not the issue you brought up.

[BowserKingKoopa]

I still don't understand why WebJobs works fine but Functions doesn't. Isn't Functions built on WebJobs? How did Functions go and manage to break a fundamental part of .NET thats always worked?

[fabiocav]

@BowserKingKoopa The key difference between the two is the fact that with WebJobs, you own the host, while with Azure Functions, you do not and the host configuration is not something that can be modified when using the consumption plan.

You have the ability to run the Azure Functions runtime where you control the host outside of consumption, which would give you full access the the host config and ability to manage binding redirects, but that does take you away from the "serverless" execution and billing model. It is, however, an option that some customers have pursued.

[tjrobinson]

Fabio, do you have any documentation for doing that? Are you referring to hosting in an App Service plan, or something else?

On 15 Feb 2018 22:30, "Fabio Cavalcante" notifications@github.com wrote:

@BowserKingKoopa https://github.com/bowserkingkoopa The key difference between the two is the fact that with WebJobs, you own the host, while with Azure Functions, you do not and the host configuration is not something that can be modified when using the consumption plan.

You have the ability to run the Azure Functions runtime where you control the host outside of consumption, which would give you full access the the host config and ability to manage binding redirects, but that does take you away from the "serverless" execution and billing model. It is, however, an option that some customers have pursued.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Azure/azure-functions-host/issues/992#issuecomment-366083619, or mute the thread https://github.com/notifications/unsubscribe-auth/AA4ef_-gau8N7-WQibPrZ7dT_VMOuUMeks5tVLAagaJpZM4K9N-C .

[wimagee]

@fabiocav that's interesting. I hadn't realised it would be different running under app service plan rather than consumption. We'd lose a key benefit over webjobs as you say, but still ..