search

Azure / azure-functions-host

The host/runtime that powers Azure Functions
https://functions.azure.com
MIT License
1.92k stars 442 forks source link

ArgumentNullException thrown while reading function metadata during host shutdown #3135

Open brettsam opened 6 years ago

brettsam commented 6 years ago

There's a chance that we throw an exception while reading function metadata while the host is stopping. Before everything is fully initialized, we see a file change, and begin stopping the host. The code below can throw because no functionName can be determined.

https://github.com/Azure/azure-functions-host/blob/bf91f5d6a585428138f273c1787f8c18af0e8eaf/src/WebJobs.Script/Host/ScriptHost.cs#L280

The trace looks like:

System.ArgumentNullException : Value cannot be null.
Parameter name: key
   at System.Collections.Generic.Dictionary`2.FindEntry(TKey key)
   at System.Collections.Generic.Dictionary`2.TryGetValue(TKey key,TValue& value)
   at Microsoft.Azure.WebJobs.Script.ScriptHost.AddFunctionError(Dictionary`2 functionErrors,String functionName,String error,Boolean isFunctionShortName) at C:\projects\azure-webjobs-sdk-script\src\WebJobs.Script\Host\ScriptHost.cs : 271
   at Microsoft.Azure.WebJobs.Script.ScriptHost.ReadFunctionMetadata(String functionDirectory,TraceWriter traceWriter,ILogger logger,Dictionary`2 functionErrors,ScriptSettingsManager settingsManager,IEnumerable`1 functionWhitelist) at C:\projects\azure-webjobs-sdk-script\src\WebJobs.Script\Host\ScriptHost.cs : 1152
   at Microsoft.Azure.WebJobs.Script.ScriptHost.ReadFunctionMetadata(IEnumerable`1 functionDirectories,TraceWriter traceWriter,ILogger logger,Dictionary`2 functionErrors,ScriptSettingsManager settingsManager,IEnumerable`1 functionWhitelist) at C:\projects\azure-webjobs-sdk-script\src\WebJobs.Script\Host\ScriptHost.cs : 1096
   at Microsoft.Azure.WebJobs.Script.ScriptHost.LoadFunctionMetadata() at C:\projects\azure-webjobs-sdk-script\src\WebJobs.Script\Host\ScriptHost.cs : 667
   at Microsoft.Azure.WebJobs.Script.ScriptHost.Initialize() at C:\projects\azure-webjobs-sdk-script\src\WebJobs.Script\Host\ScriptHost.cs : 298
   at Microsoft.Azure.WebJobs.Script.ScriptHostManager.RunAndBlock(CancellationToken cancellationToken) at C:\projects\azure-webjobs-sdk-script\src\WebJobs.Script\Host\ScriptHostManager.cs : 177

Note that this is not the cause of the host stopping, and everything continues on as expected with the new host. However, it is noisy and can cause alerts in customers Application Insights accounts due to the exception firing. We've had at least one CRI due to this.

ZivShasha commented 6 years ago

When will this bug get into progress? I had another occurrence of it today

fabiocav commented 5 years ago

Need to check if this impacts V2 as well.

alrod commented 4 years ago

Also need check the restart code paths. We had a incident.