search

Azure / azure-functions-host

The host/runtime that powers Azure Functions
https://functions.azure.com
MIT License
1.92k stars 442 forks source link

Unable to run Durable Functions with Azure Storage Emulator v5.8 #3795

Open csheets opened 5 years ago

csheets commented 5 years ago

I was successfully running Durable Functions for a few weeks but now all of a sudden I am not able to run them and am getting the following output. Not sure what has changed -- any pointers on what to look at would be most appreciated. I have also included a related stacktrace below.

Azure Functions Core Tools (2.1.725 Commit hash: 68f448fe6a60e1cade88c2004bf6491
af7e5f1df)
Function Runtime Version: 2.0.12134.0
[10/14/2018 10:17:59 PM] Building host: startup suppressed:False, configuration
suppressed: False
[10/14/2018 10:17:59 PM] Reading host configuration file 'C:\Users\csheets\sourc
e\repos\FunctionApp4\FunctionApp1\bin\Debug\netstandard2.0\host.json'
[10/14/2018 10:17:59 PM] Host configuration file read:
[10/14/2018 10:17:59 PM] {
[10/14/2018 10:17:59 PM]   "version": "2.0"
[10/14/2018 10:17:59 PM] }
[10/14/2018 10:18:38 PM] A host error has occurred
[10/14/2018 10:18:38 PM] Microsoft.WindowsAzure.Storage: Server encountered an i
nternal error. Please try again after some time.
Listening on http://0.0.0.0:7071/
Hit CTRL-C to exit...

The following is a stacktrace that is being thrown:

System.AggregateException
  HResult=0x80131500
  Message=One or more errors occurred.
  Source=System.Private.CoreLib
  StackTrace:
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at Microsoft.WindowsAzure.Storage.Blob.BlobWriteStream.<Dispose>b__8_0()
   at Microsoft.WindowsAzure.Storage.Core.Util.CommonUtility.RunWithoutSynchronizationContext(Action actionToRun)
   at Microsoft.WindowsAzure.Storage.Blob.BlobWriteStream.Dispose(Boolean disposing)
   at System.IO.Stream.Close()
   at System.IO.StreamWriter.Dispose(Boolean disposing)
   at System.IO.TextWriter.Dispose()
   at Microsoft.Azure.WebJobs.Script.WebHost.BlobStorageSecretsRepository.<WriteToBlobAsync>d__22.MoveNext() in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\Security\KeyManagement\BlobStorageSecretsRepository.cs:line 167
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at Microsoft.Azure.WebJobs.Script.WebHost.BlobStorageSecretsRepository.<WriteAsync>d__18.MoveNext() in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\Security\KeyManagement\BlobStorageSecretsRepository.cs:line 128
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at Microsoft.Azure.WebJobs.Script.WebHost.SecretManager.<PersistSecretsAsync>d__26`1.MoveNext() in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\Security\KeyManagement\SecretManager.cs:line 412
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at Microsoft.Azure.WebJobs.Script.WebHost.SecretManager.<GetHostSecretsAsync>d__11.MoveNext() in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\Security\KeyManagement\SecretManager.cs:line 82
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at Microsoft.Azure.WebJobs.Script.WebHost.DefaultScriptWebHookProvider.<GetOrCreateExtensionKey>d__6.MoveNext() in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\WebHooks\DefaultScriptWebHookProvider.cs:line 72
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at Microsoft.Azure.WebJobs.Script.WebHost.DefaultScriptWebHookProvider.GetExtensionWebHookRoute(String extensionName) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\WebHooks\DefaultScriptWebHookProvider.cs:line 64
   at Microsoft.Azure.WebJobs.Script.WebHost.DefaultScriptWebHookProvider.GetUrl(IExtensionConfigProvider extension) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\WebHooks\DefaultScriptWebHookProvider.cs:line 49
   at Microsoft.Azure.WebJobs.Host.Config.ExtensionConfigContext.GetWebhookHandler() in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Config\ExtensionConfigContext.cs:line 55
   at Microsoft.Azure.WebJobs.Extensions.DurableTask.DurableTaskExtension.Microsoft.Azure.WebJobs.Host.Config.IExtensionConfigProvider.Initialize(ExtensionConfigContext context)
   at Microsoft.Azure.WebJobs.Host.DefaultExtensionRegistryFactory.Create() in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\DefaultExtensionRegistryFactory.cs:line 38
   at Microsoft.Azure.WebJobs.WebJobsServiceCollectionExtensions.<>c.<AddWebJobs>b__1_0(IServiceProvider p) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Hosting\WebJobsServiceCollectionExtensions.cs:line 50
   at DryIoc.Microsoft.DependencyInjection.DryIocAdapter.<>c__DisplayClass3_0.<RegisterDescriptor>b__0(IResolverContext r) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\DryIocAdapter.cs:line 156
   at DryIoc.Registrator.<>c__DisplayClass27_0.<RegisterDelegate>b__0(IResolverContext r) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 4530
   at DryIoc.Factory.<>c__DisplayClass26_1.<ApplyReuse>b__2() in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6584
   at DryIoc.Scope.TryGetOrAdd(ImMap`1 items, Int32 id, CreateScopedValue createValue, Int32 disposalOrder) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 7825
   at DryIoc.Scope.GetOrAdd(Int32 id, CreateScopedValue createValue, Int32 disposalOrder) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 7810
   at DryIoc.Factory.ApplyReuse(Expression serviceExpr, Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6583
   at DryIoc.Factory.GetExpressionOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6544
   at DryIoc.Factory.GetDelegateOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6611
   at DryIoc.DelegateFactory.GetDelegateOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 7719
   at DryIoc.Container.DryIoc.IResolver.Resolve(Type serviceType, Object serviceKey, IfUnresolved ifUnresolved, Type requiredServiceType, Request preResolveParent, Object[] args) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 290
   at DryIoc.Factory.<>c__DisplayClass26_1.<ApplyReuse>b__2() in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6584
   at DryIoc.Scope.TryGetOrAdd(ImMap`1 items, Int32 id, CreateScopedValue createValue, Int32 disposalOrder) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 7825
   at DryIoc.Scope.GetOrAdd(Int32 id, CreateScopedValue createValue, Int32 disposalOrder) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 7810
   at DryIoc.Factory.ApplyReuse(Expression serviceExpr, Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6583
   at DryIoc.Factory.GetExpressionOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6544
   at DryIoc.Factory.GetDelegateOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6611
   at DryIoc.Container.ResolveAndCacheDefaultFactoryDelegate(Type serviceType, IfUnresolved ifUnresolved) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 210
   at DryIoc.Container.DryIoc.IResolver.Resolve(Type serviceType, IfUnresolved ifUnresolved) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 195
   at Microsoft.Azure.WebJobs.Script.WebHost.DependencyInjection.JobHostServiceProvider.GetService(Type serviceType, IfUnresolved ifUnresolved) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\JobHostServiceProvider.cs:line 96
   at Microsoft.Azure.WebJobs.Script.WebHost.DependencyInjection.JobHostServiceProvider.GetRequiredService(Type serviceType) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\JobHostServiceProvider.cs:line 79
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at Microsoft.Azure.WebJobs.WebJobsServiceCollectionExtensions.<>c.<AddWebJobs>b__1_4(IServiceProvider p) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Hosting\WebJobsServiceCollectionExtensions.cs:line 83
   at DryIoc.Microsoft.DependencyInjection.DryIocAdapter.<>c__DisplayClass3_0.<RegisterDescriptor>b__0(IResolverContext r) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\DryIocAdapter.cs:line 156
   at DryIoc.Registrator.<>c__DisplayClass27_0.<RegisterDelegate>b__0(IResolverContext r) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 4530
   at DryIoc.Factory.<>c__DisplayClass26_1.<ApplyReuse>b__2() in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6584
   at DryIoc.Scope.TryGetOrAdd(ImMap`1 items, Int32 id, CreateScopedValue createValue, Int32 disposalOrder) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 7825
   at DryIoc.Scope.GetOrAdd(Int32 id, CreateScopedValue createValue, Int32 disposalOrder) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 7810
   at DryIoc.Factory.ApplyReuse(Expression serviceExpr, Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6583
   at DryIoc.Factory.GetExpressionOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6544
   at DryIoc.Factory.GetDelegateOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6611
   at DryIoc.DelegateFactory.GetDelegateOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 7719
   at DryIoc.Container.DryIoc.IResolver.Resolve(Type serviceType, Object serviceKey, IfUnresolved ifUnresolved, Type requiredServiceType, Request preResolveParent, Object[] args) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 290
   at DryIoc.Factory.<>c__DisplayClass26_1.<ApplyReuse>b__2() in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6584
   at DryIoc.Scope.TryGetOrAdd(ImMap`1 items, Int32 id, CreateScopedValue createValue, Int32 disposalOrder) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 7825
   at DryIoc.Scope.GetOrAdd(Int32 id, CreateScopedValue createValue, Int32 disposalOrder) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 7810
   at DryIoc.Factory.ApplyReuse(Expression serviceExpr, Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6583
   at DryIoc.Factory.GetExpressionOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6544
   at DryIoc.ReflectionFactory.CreateExpressionOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 7059
   at DryIoc.Factory.GetExpressionOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6534
   at DryIoc.ReflectionFactory.CreateExpressionOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 7059
   at DryIoc.Factory.GetExpressionOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6534
   at DryIoc.Factory.GetDelegateOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6611
   at DryIoc.Container.ResolveAndCacheDefaultFactoryDelegate(Type serviceType, IfUnresolved ifUnresolved) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 210
   at DryIoc.Container.DryIoc.IResolver.Resolve(Type serviceType, IfUnresolved ifUnresolved) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 195
   at Microsoft.Azure.WebJobs.Script.WebHost.DependencyInjection.JobHostServiceProvider.GetService(Type serviceType, IfUnresolved ifUnresolved) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\JobHostServiceProvider.cs:line 96
   at Microsoft.Azure.WebJobs.Script.WebHost.DependencyInjection.JobHostServiceProvider.GetRequiredService(Type serviceType) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\JobHostServiceProvider.cs:line 79
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at Microsoft.Azure.WebJobs.Script.ScriptHostBuilderExtensions.<>c.<AddScriptHostCore>b__2_5(IServiceProvider p) in C:\azure-webjobs-sdk-script\src\WebJobs.Script\ScriptHostBuilderExtensions.cs:line 115
   at DryIoc.Microsoft.DependencyInjection.DryIocAdapter.<>c__DisplayClass3_0.<RegisterDescriptor>b__0(IResolverContext r) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\DryIocAdapter.cs:line 156
   at DryIoc.Registrator.<>c__DisplayClass27_0.<RegisterDelegate>b__0(IResolverContext r) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 4530
   at DryIoc.Factory.<>c__DisplayClass26_1.<ApplyReuse>b__2() in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6584
   at DryIoc.Scope.TryGetOrAdd(ImMap`1 items, Int32 id, CreateScopedValue createValue, Int32 disposalOrder) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 7825
   at DryIoc.Scope.GetOrAdd(Int32 id, CreateScopedValue createValue, Int32 disposalOrder) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 7810
   at DryIoc.Factory.ApplyReuse(Expression serviceExpr, Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6583
   at DryIoc.Factory.GetExpressionOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6544
   at DryIoc.Factory.GetDelegateOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6611
   at DryIoc.DelegateFactory.GetDelegateOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 7719
   at DryIoc.Container.ResolveAndCacheDefaultFactoryDelegate(Type serviceType, IfUnresolved ifUnresolved) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 210
   at DryIoc.Container.DryIoc.IResolver.Resolve(Type serviceType, IfUnresolved ifUnresolved) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 195
   at Microsoft.Azure.WebJobs.Script.WebHost.DependencyInjection.JobHostServiceProvider.GetService(Type serviceType, IfUnresolved ifUnresolved) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\JobHostServiceProvider.cs:line 96
   at Microsoft.Azure.WebJobs.Script.WebHost.DependencyInjection.JobHostServiceProvider.GetRequiredService(Type serviceType) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\JobHostServiceProvider.cs:line 79
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at Azure.Functions.Cli.Actions.HostActions.StartHostAction.<RunAsync>d__44.MoveNext()

Inner Exception 1:
StorageException: Server encountered an internal error. Please try again after some time.
csheets commented 5 years ago

I switched from using local storage emulator to using Azure StorageAccount and the function app run ok -- so it is something to do with the Azure Storage emulator but I have no idea what. I have attempted to uninstall and re-install the storage emulator but the same error occurs after the uninstall/re-install process.

cgillum commented 5 years ago

Which version of the storage emulator are you using? This actually doesn't appear to be a Durable Functions issue, but rather an issue with the Azure Functions host itself, which uses storage for things like key management (key management frames appear in your callstack).

csheets commented 5 years ago

I am using version: Windows Azure Storage Emulator 5.8.0.0 command line tool

I am able to run "regular" Azure Functions locally without any issues. Seems to be just Durable Functions that are having the problem.

cgillum commented 5 years ago

Thanks - I downloaded 5.8 and observed the same problem. I have no idea why the Durable extension seems to be triggering this issue because the code path seems completely unrelated. I'll follow up with the Azure Functions Host team. I think this might possibly be a problem with extensions that use web hooks.

Workaround

Here is a workaround I identified from the Functions Key Management documentation. In local.settings.json, add a new setting called AzureWebJobsSecretStorageType and set it to "files".

{
  "IsEncrypted": false,
  "Values": {
    "AzureWebJobsStorage": "UseDevelopmentStorage=true",
    "FUNCTIONS_WORKER_RUNTIME": "dotnet",
    "AzureWebJobsSecretStorageType": "files"
  }
}

/cc @fabiocav

csheets commented 5 years ago

ok - thanks

prior to your reply i ended up uninstalling and re-installing VS 2017 and that resolved the issue -- looks like it re-installed Storage Emulator 5.6 in the process so just going to stay there for now

cgillum commented 5 years ago

Good to know there is a way to go back (I might consider doing that as well).

I've also opened a bug in the Azure-Functions-Host repo to track this since it will likely affect more than just Durable Functions: https://github.com/Azure/azure-functions-host/issues/3621

fabiocav commented 5 years ago

Indeed, this is not specific to Durable Functions, but on a host code path it relies on.

This looks like an issue with the more recent version of the Emulator, we'll investigate.

tsubakimoto commented 5 years ago

I also encountered the same error. Also, the problem is temporarily resolved by the method @cgillum presented. Thanks. https://github.com/Azure/azure-functions-durable-extension/issues/474#issuecomment-430337085

I expect that the error will be resolved.

fabiocav commented 5 years ago

Moving this to host repo for triage and assignment

apsteger commented 5 years ago

I experienced this issue after installing the Visual Studio 2019 Preview. Installing Storage Emulator 5.9 fixed it for me.

iwaldman commented 5 years ago

I experienced the same issue. Thank you @cgillum for the workaround.

opyke commented 5 years ago

I am also experiencing this issue. Using the workaround.

MaximRouiller commented 5 years ago

UPDATE: said blog post

Alright. I'll be writing a blog post about this issue to make it more visible but basically, updating Storage Emulator to 5.9 solves your issue.

I was running Azure Durable Functions on .NET and NodeJS with the workaround for my samples:

I was running the v5.8 of Azure Storage Emulator. No idea what changed since the docs isn't updated yet. I've opened an issue to ensure that we get some notes.

So... updating to v5.9 fixed the issue in .NET as well as with NodeJS using the Windows Storage Emulator.

My Recommendation

Leave the workaround in place. People may have the 5.8 version which could lead to issues. There's no clear message that the parameter is missing which makes it hard to debug. Leaving the parameter in place doesn't seem to be causing any issues with 5.9 either.

Follow-Up

@fabiocav @cgillum Any point in making the message clearer for the users? Basically, 5.7 works, 5.9 works, 5.8 🔥.

Obviously not an issue from Azure Functions point of view but it does look more like a breaking change in the Emulator's API.

Thoughts?

chrislangston commented 5 years ago

Same thing is happening even if you have VS 2019 Installed and running Azure Storage Emulator 5.9. I had to add this entry to the local.settings.json to get the Azure Function to work.

MaximRouiller commented 5 years ago

@chrislangston Which version of Azure Functions are you using?

If you run func on your CLI, can you copy the lines below the magical lightning? Mine looks like this:

Azure Functions Core Tools (2.7.1149 Commit hash: 476884e4a09c54686e460fd0c050d15139bf073e)
Function Runtime Version: 2.0.12438.0
yojagad commented 5 years ago

Adding this to triaged to verify whether there are issues when running with Storage Emulator 5.9.

r0bturner commented 4 years ago

I'm still unable to run durable functions with Storage Emulator 5.10 and AzureWebJobsSecretStorageType set to "files" in the local.settings.json file.

If I remove the AzureWebJobsSecretStorageType setting (so it defaults to blobs) I get this exception in the Storage Account logs:

03/01/2020 09:53:22 [Error_debug] [ActivityId=919f1fa3-9cd5-441d-a07a-411fec2f6828] Exception raised during processing of request.
 Microsoft.Cis.Services.Nephos.Common.Protocols.Rest.FatalServerCrashingException: 
The fatal unexpected exception 'Exception has been thrown by the target of an invocation.' encountered during processing of request. 
---> System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. 
---> System.InvalidOperationException: This implementation is not part of the Windows Platform FIPS validated cryptographic algorithms.
   at System.Security.Cryptography.MD5CryptoServiceProvider..ctor()

This means I can't upload blobs to the storage emulator. I'm unable to disable FIPS compliant algorithms for encryption because i'm using a work machine. Looks like Durable Functions rely on blobs in the azure-webjobs-hosts and azure-webjobs-secrets containers. It's just a shame the AzureWebJobsSecretStorageType setting set to "files" doesn't overcome this issue. Wonder if this would be possible to fix in the future?

Of course I can workaround this by using a real storage account instead of the emulator..

HQJaTu commented 4 years ago

Looks like Durable Functions rely on blobs in the azure-webjobs-hosts and azure-webjobs-secrets containers. It's just a shame the AzureWebJobsSecretStorageType setting set to "files" doesn't overcome this issue. Wonder if this would be possible to fix in the future?

Of course I can workaround this by using a real storage account instead of the emulator..

Actually, I failed to successfully deploy my Durable Functions into Azure because of this same thing. Manual removal of the blob's contents (the keys) solved the issue.

vinayakmoh commented 4 years ago

Thanks - I downloaded 5.8 and observed the same problem. I have no idea why the Durable extension seems to be triggering this issue because the code path seems completely unrelated. I'll follow up with the Azure Functions Host team. I think this might possibly be a problem with extensions that use web hooks.

Workaround

Here is a workaround I identified from the Functions Key Management documentation. In local.settings.json, add a new setting called AzureWebJobsSecretStorageType and set it to "files".

{
  "IsEncrypted": false,
  "Values": {
    "AzureWebJobsStorage": "UseDevelopmentStorage=true",
    "FUNCTIONS_WORKER_RUNTIME": "dotnet",
    "AzureWebJobsSecretStorageType": "files"
  }
}

/cc @fabiocav

This solved issue for me. I had upgraded to VS 2019 and Emulator 5.1

HQJaTu commented 4 years ago

Yes, that will solve the problem for local dev. Deleting the Blobs in question will solve it when the same thing happens in Azure.

holladeerun commented 1 year ago

This just saved me some time with Azurite