Closed dsm0880 closed 6 years ago
I have the same issue here. Function App worked fine with ~1, but when I switched to beta, it gives me the error Service Unavailable and I can not call any of the Functions anymore. I was trying out the Function App Chaining using Durable Functions.
@tsmccartan Haven't received any responses yet, but we are still investigating on our end. It was super weird basically someone would add a single function and it would break everything by getting the message I posted above. Are you getting the same error when you go to a specific function? I got the "Service Unavailable" if I went to the function app url. If I reverted back and removed the newly added function everything would work as expected.
On a somewhat related note if you are having any trouble we got remote debugging to work by attaching to dotnet.exe instead of the w3wp.exe which everyone says to do.
@dsm0880 It generally tells me that the Service in unavailable, when I go to a specific Function I get the following (Which makes me think that maybe the published functions are not compatible with the assemblies for 2.x?). I did not redeploy any of the functions. I have several Azure Functions that use KeyVault, ServiceBus, etc. I was testing Durable Functions and added the NuGet Package and the sample functions, so I can chain together some Functions. I found out that when I tested it, they did not work. I looked at the docs, and it said the Function Apps had to be configured for 2.0 (Beta), so I switched it, and after that none of the functions worked anymore? I have restarted, stopped, started, etc.
Directions Followed to Target: https://docs.microsoft.com/en-us/azure/azure-functions/set-runtime-version
ERROR The function runtime is unable to start. System.Private.CoreLib: Exception has been thrown by the target of an invocation. Microsoft.ServiceBus: The type initializer for 'Microsoft.ServiceBus.Messaging.Constants' threw an exception. Microsoft.ServiceBus: Could not load file or assembly 'System.ServiceModel, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'. The system cannot find the file specified.
@tsmccartan You may have a different issue, assuming everything compiles fine locally? Ensure you are actually running with V2 of core tools. If you go to %localappdata% in file explorer do you see Azure.Functions.V2.Cli? If not make sure you have installed the latest azure functions extension in VS Extensions and Updates. After that make sure you have <AzureFunctionsVersion>v2</AzureFunctionsVersion>
in the csproj file. Then recompile and see if you get any errors.
@dsm0880 Thanks for the information. I definitely did not do that, as my project still targets
Thanks for all your quick feedback, sorry my issue is not exactly like yours and won't be able to help with yours.
@dsm0880 I do get the same error and have the latest Azure Functions extension installed in VS and the csproj file also looks fine. The error is
The function runtime is unable to start. System.Private.CoreLib: Exception has been thrown by the target of an invocation. Microsoft.ServiceBus: The type initializer for 'Microsoft.ServiceBus.Messaging.Constants' threw an exception. Microsoft.ServiceBus: Could not load file or assembly 'System.ServiceModel, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'. The system cannot find the file specified. Session Id: 3e83a74f5a254b328ff24110b66f2096
Timestamp: 2018-03-14T15:58:11.358Z
We did a new runtime update this week that addresses some of the issues above, however the we're still updating Durable Functions preview to work with that new release. @ConnorMcMahon can provide an update for that.
@davidebbo
As I mentioned in the other thread, we're having this issue at the top of this thread with the new runtime and it's blocking us.
As requested, here are the function details
East Region 2018-03-30T00:17:36.960 Function started (Id=13990dee-f7fe-42a8-ab5e-a55448714ac8)
@mayoatbm Some questions:
@davidebbo
We have six function apps impacted by this issue. I tried upgrading one of them per your instructions and i got the error at the top of this thread. Unfortunately i can't share the details because I can't it to run to capture that information ((host is not running). I can send it to you privately if you'd like.
@mayoatbm looking at our logs, I see a very different error in Azure:
System.MissingMethodException : Method not found: 'Void Microsoft.Azure.WebJobs.Host.Config.FluentBindingRule`1.BindToInput(System.Func`2<!0,!!0>)'.
at BNSF.UAS.Platform.IoTHubManager.DI.DependencyConfiguration.Initialize(ExtensionConfigContext context)
at Microsoft.Azure.WebJobs.Host.Executors.JobHostConfigurationExtensions.InvokeExtensionConfigProviders(ExtensionConfigContext context) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\JobHostConfigurationExtensions.cs : 421
at Microsoft.Azure.WebJobs.Host.Executors.JobHostConfigurationExtensions.CreateStaticServices(JobHostConfiguration config) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\JobHostConfigurationExtensions.cs : 112
at Microsoft.Azure.WebJobs.JobHost.InitializeServices() at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\JobHost.cs : 470
at Microsoft.Azure.WebJobs.JobHost.get_Services() at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\JobHost.cs : 482
at Microsoft.Azure.WebJobs.Script.Utility.CreateMetadataProvider(JobHost host) at C:\azure-webjobs-sdk-script\src\WebJobs.Script\Utility.cs : 372
at Microsoft.Azure.WebJobs.Script.ScriptHost.LoadBindingExtensions(IEnumerable`1 functionMetadata,JObject hostConfigObject) at C:\azure-webjobs-sdk-script\src\WebJobs.Script\Host\ScriptHost.cs : 849
at Microsoft.Azure.WebJobs.Script.ScriptHost.Initialize() at C:\azure-webjobs-sdk-script\src\WebJobs.Script\Host\ScriptHost.cs : 303
at Microsoft.Azure.WebJobs.Script.ScriptHostManager.RunAndBlock(CancellationToken cancellationToken) at C:\azure-webjobs-sdk-script\src\WebJobs.Script\Host\ScriptHostManager.cs : 194
This suggests that you have your own custom extension that was built against the previous runtime, so it's hitting a method that's now gone (or changed). You will need to rebuild your extension against the latest packages.
@davidebbo
That's the error we get with the new runtime using our old code (make sense). The problem is that upgrading gives us the abstractions error. There's another function app in that same resource group that has the new nuget packages and sdk per your instructions. That gives the error at the top of this thread.
Does that clarify things?
@davidebbo This issue might also be related.
@mayoatbm yes, that helps. So to summarize:
MissingMethodException
Microsoft.AspNetCore.Mvc.Abstractions
error.Problem may be that VS still needs to take an update. For now, stay with old build, and we'll figure this out.
/cc @fabiocav
@davidebbo
Great.
Yes, that's a good summary. The only clarification I will make is that we get this "abstractions" error in Visual Studio and in the Core Runtime from npm, and in the cloud. So it's everywhere.
Can you guarantee that the old version of the runtime will be available until this issue is resolved?
Also, what is the likelihood that this situation (deploying a runtime with known breaking changes) happens again? We know that functions v2 is in beta but sudden runtime failures in Azure are a whole different ballgame than the other (nuget version dependencies, sdk vs cli, vs visual studio extension, function generation etc) issues we've seen.
Thank you for your responsiveness.
@mayoatbm can you try the following:
dotnet publish
bin\Debug\netstandard2.0
func host start
(get the latest from npm, e.g. npm i -g azure-functions-core-tools@core
)@mayoatbm Just saw your other reply
@davidebbo
The new cli works. Thank you very much, that was fast. Unfortunately the function still doesn't work in azure (i deployed the output of dotnet publish to via the kudu console). I still get the abstractions exception there.
Also. Thank you for the assurances re: keeping the old runtime around and future upgrades. That helps with the level of concern going forward.
I also appreciate the fast, patient responses. Please keep us posted in this thread on when the azure part is fixed so we can perform the upgrade. Let me know if i can provide any additional information.
@mayoatbm is that still in the same app as I was looking at yesterday, or a different one? I do not see any updates on that one.
It's the other one in that same resource group. Remember I couldn't get you the function details because the host isn't running.
@mayoatbm got it. I just took a look, but I'm seeing yet a different exception on that one:
System.ArgumentNullException : Value cannot be null.
Parameter name: value
at Microsoft.ApplicationInsights.Extensibility.TelemetryConfiguration.set_InstrumentationKey(String value)
at BNSF.UAS.Platform.DataFabric.DI.DependencyConfiguration.SetupApplicationInsights() at C:\Projects\BNSF\Code\BMRepo\UAV Analytics MVP\bnsf-uas-platform\DataFabric\DataFabric\DI\DependencyConfiguration.cs : 116
at BNSF.UAS.Platform.DataFabric.DI.DependencyConfiguration.SetupLogging(ExtensionConfigContext context,ContainerBuilder builder) at C:\Projects\BNSF\Code\BMRepo\UAV Analytics MVP\bnsf-uas-platform\DataFabric\DataFabric\DI\DependencyConfiguration.cs : 100
at BNSF.UAS.Platform.DataFabric.DI.DependencyConfiguration.RegisterServices(ExtensionConfigContext context) at C:\Projects\BNSF\Code\BMRepo\UAV Analytics MVP\bnsf-uas-platform\DataFabric\DataFabric\DI\DependencyConfiguration.cs : 71
at BNSF.UAS.Platform.DataFabric.DI.DependencyConfiguration.Startup(ExtensionConfigContext context) at C:\Projects\BNSF\Code\BMRepo\UAV Analytics MVP\bnsf-uas-platform\DataFabric\DataFabric\DI\DependencyConfiguration.cs : 43
at BNSF.UAS.Platform.DataFabric.DI.DependencyConfiguration.Initialize(ExtensionConfigContext context) at C:\Projects\BNSF\Code\BMRepo\UAV Analytics MVP\bnsf-uas-platform\DataFabric\DataFabric\DI\DependencyConfiguration.cs : 34
at Microsoft.Azure.WebJobs.Host.Executors.JobHostConfigurationExtensions.InvokeExtensionConfigProviders(ExtensionConfigContext context) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\JobHostConfigurationExtensions.cs : 421
at Microsoft.Azure.WebJobs.Host.Executors.JobHostConfigurationExtensions.CreateStaticServices(JobHostConfiguration config) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\JobHostConfigurationExtensions.cs : 112
at Microsoft.Azure.WebJobs.JobHost.InitializeServices() at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\JobHost.cs : 470
at Microsoft.Azure.WebJobs.JobHost.get_Services() at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\JobHost.cs : 482
at Microsoft.Azure.WebJobs.Script.Utility.CreateMetadataProvider(JobHost host) at C:\azure-webjobs-sdk-script\src\WebJobs.Script\Utility.cs : 372
at Microsoft.Azure.WebJobs.Script.ScriptHost.LoadBindingExtensions(IEnumerable`1 functionMetadata,JObject hostConfigObject) at C:\azure-webjobs-sdk-script\src\WebJobs.Script\Host\ScriptHost.cs : 849
at Microsoft.Azure.WebJobs.Script.ScriptHost.Initialize() at C:\azure-webjobs-sdk-script\src\WebJobs.Script\Host\ScriptHost.cs : 303
at Microsoft.Azure.WebJobs.Script.ScriptHostManager.RunAndBlock(CancellationToken cancellationToken) at C:\azure-webjobs-sdk-script\src\WebJobs.Script\Host\ScriptHostManager.cs : 194
@davidebbohttps://github.com/davidebbo
That's my fault. The app insights key was missing (I didn't copy over the app setting when I deployed via kudu). I just put it in there and thing seem to be working, which is great. I must have misread the timestamp on the error below (from our app insights) after I deployed it. Apologies for the subsequent confusion.
So, is this where we stand? Using the CLI to run locally works and dotnet publish with a manual deploy to azure works. But anything through Visual studio won't work until the extension is updated?
Thanks!
@mayoatbm yes, that's exactly where things stand. The VS extension update is hopefully the last step. @soninaren should be able to provide update on the progress.
@davidebbo
Great. We will keep an eye out for it. Thanks again for all your help, we're in a much better than we were yesterday.
@davidebbo I had a successfully running Durable Function as well as other functions running in Azure that was processing data. but these are now all failing in Azure with the following error: (I Get this error whenever I click on the function in the Azure Portal. When I run the function from Postman I get "Function host is not running")
Error: The function runtime is unable to start. Microsoft.Azure.WebJobs.Extensions.DurableTask: Method not found: 'Microsoft.Azure.WebJobs.Host.TraceWriter Microsoft.Azure.WebJobs.Host.Config.ExtensionConfigContext.get_Trace()'.
This is using the Beta version and DotNetCore. I didn't change any of my code that was running on a 5 minute timer function, but found out today that it has been failing every time now? Is this a known issue (As you stated above, there are breaking changes and it is NOT working with Durable Functions)? Is there anything I can do, to get this back processing? Or do I just need to wait for some update?
Thanks!
@tsmccartan please see https://github.com/Azure/app-service-announcements/issues/94.
@davidebbo Thanks, this solved my issues and my functions are back operational, processing data! Do you know if this addresses the issues with creating Custom Telemetry in Applications Insights?
From this Post: https://github.com/Azure/azure-functions-host/issues/2473#issuecomment-376984633
UPDATE - I tried to add in the following code again, and it still fails with the following error.
EXCEPTION Could not load file or assembly 'Microsoft.ApplicationInsights, Version=2.5.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35'. Could not find or load a specific file. (Exception from HRESULT: 0x80131621) Could not load file or assembly 'Microsoft.ApplicationInsights, Version=2.5.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35'.
I have the following NuGet Packages Referenced:
<PackageReference Include="Microsoft.ApplicationInsights" Version="2.5.1" />
<PackageReference Include="Microsoft.ApplicationInsights.PerfCounterCollector" Version="2.5.1" />
<PackageReference Include="Microsoft.Azure.WebJobs.Extensions" Version="3.0.0-beta5" />
<PackageReference Include="Microsoft.Azure.WebJobs.Extensions.DurableTask" Version="1.2.2-beta3" />
<PackageReference Include="Microsoft.Azure.WebJobs.Extensions.Http" Version="3.0.0-beta5" />
<PackageReference Include="Microsoft.Extensions.Configuration.AzureKeyVault" Version="2.0.0" />
<PackageReference Include="System.Data.SqlClient" Version="4.4.0" />
It took a little longer than we would have wanted, but the updated Azure Functions and WebJobs Tools VS extension is now up. Please go ahead and upgrade. The new version is 15.0.40405.0.
Thanks for your patience on this.
Had same issue after deploying .NET Azure Function in "Azure Function App", which was previously hosting .NET Core Azure Function. I have created a new Azure Function App and deployed .NET Azure Function. All worked fine.
Please note, I have checked "delete files" before deployment. It didn't help.
When publishing to a Function app I can no longer run my functions. If I completely delete the function app and recreate it, provision a simple HTTP Trigger template everything works. After I publish I receive the following error when accessing an individual function:
The function runtime is unable to start. System.Private.CoreLib: Could not load file or assembly 'Microsoft.AspNetCore.Mvc.Abstractions, Version=2.0.2.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. Could not find or load a specific file. (Exception from HRESULT: 0x80131621). System.Private.CoreLib: Could not load file or assembly 'Microsoft.AspNetCore.Mvc.Abstractions, Version=2.0.2.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. Session Id: d1a1852c325d47cd9e76a32ddc097293
Timestamp: 2018-02-22T21:38:33.235Z
I also notice I cannot access the host keys when going to the root function app. I get the following error:
Service Unavailable - https://c365-funcitons2.azurewebsites.net/admin/host/keys Session Id: d1a1852c325d47cd9e76a32ddc097293
Timestamp: 2018-02-22T21:42:10.525Z
Investigative information
Please provide the following:
Repro steps
Provide the steps required to reproduce the problem:
Known workarounds
I don't have any workarounds and I consider this a pretty bad issue for myself. I can tell you a related ticket had the same issue where the runtime had a similar error regarding runtime not being able to find the MVC.Abstractions.dll. Since it was local I could manually replace the dll, but of course I cannot do this in azure.