search

Azure / azure-functions-host

The host/runtime that powers Azure Functions
https://functions.azure.com
MIT License
1.94k stars 441 forks source link

[FromServices] race condition leads to null #5161

Open mathewc opened 5 years ago

mathewc commented 5 years ago

There are several places in our code where we use [FromServices] to resolve a host level service. In certain situations, even if the action is marked [RequiresRunningHost] a null can be returned. In a recent pr here we worked around this for a particular API, with the follow up being to put in a general solution for this so we're doing this consistently across all actions.

E.g. I'm seeing exceptions in production now for the new runtime scale endpoint (HostController.GetScaleStatus) due to this issue.

System.InvalidOperationException : No service for type 'Microsoft.Azure.WebJobs.Script.IScriptJobHost' has been registered.
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider,Type serviceType)
   at Microsoft.AspNetCore.Mvc.ModelBinding.Binders.ServicesModelBinder.BindModelAsync(ModelBindingContext bindingContext)
   at async Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder.BindModelAsync(??)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.AspNetCore.Mvc.Internal.ControllerBinderDelegateProvider.<>c__DisplayClass0_0.<CreateBinderDelegate>g__Bind|0(??)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.InvokeInnerFilterAsync()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeNextResourceFilter()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Rethrow(ResourceExecutedContext context)
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Next(State& next,Scope& scope,Object& state,Boolean& isCompleted)
   at async Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeFilterPipelineAsync()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeAsync()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.AspNetCore.Builder.RouterMiddleware.Invoke(HttpContext httpContext)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.AspNetCore.Builder.RouterMiddleware.Invoke(HttpContext httpContext)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Azure.WebJobs.Script.WebHost.Middleware.FunctionInvocationMiddleware.Invoke(HttpContext context) at C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\Middleware\FunctionInvocationMiddleware.cs : 52
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Azure.WebJobs.Script.Middleware.DefaultMiddlewarePipeline.<>c.<BuildPipeline>b__7_0(??) at C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\Middleware\JobHostMiddleware\DefaultMiddlewarePipeline.cs : 31
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Azure.WebJobs.Script.WebHost.Middleware.JobHostPipelineMiddleware.Invoke(HttpContext httpContext,IJobHostMiddlewarePipeline middleware) at C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\Middleware\JobHostPipelineMiddleware.cs : 26
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Azure.WebJobs.Script.WebHost.Middleware.ResponseContextItemsCheckMiddleware.Invoke(HttpContext context) at C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\Middleware\ResponseContextItemsCheckMiddleware.cs : 29
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Azure.WebJobs.Script.WebHost.Middleware.HomepageMiddleware.Invoke(HttpContext context) at C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\Middleware\HomepageMiddleware.cs : 34
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.AspNetCore.Buffering.ResponseBufferingMiddleware.Invoke(HttpContext httpContext) at C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\Middleware\Buffering\ResponseBufferingMiddleware.cs : 38
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Azure.WebJobs.Script.WebHost.Middleware.ExceptionMiddleware.Invoke(HttpContext context) at C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\Middleware\ExceptionMiddleware.cs : 28
mathewc commented 4 years ago

@fabiocav thoughts on the right fix for this? We're seeing this issue for APIs other than Warmup that we originally logged this for. I just logged https://github.com/Azure/azure-functions-host/issues/6826. The unfortunate workaround for that issue will be to do a manual service lookup as we did for the Warmup API.

vrdmr commented 3 years ago

@fabiocav @mathewc - Ping - any updates on this?

bbondarets commented 1 year ago

We face similar issue. .NET 6.0, Microsoft.NET.Sdk.Functions Version="4.1.3"

System.InvalidOperationException: No service for type 'Microsoft.Azure.WebJobs.Script.IScriptJobHost' has been registered.
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.AspNetCore.Mvc.ModelBinding.Binders.ServicesModelBinder.BindModelAsync(ModelBindingContext bindingContext)
   at Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder.BindModelAsync(ActionContext actionContext, IModelBinder modelBinder, IValueProvider valueProvider, ParameterDescriptor parameter, ModelMetadata metadata, Object value, Object container)
   at Microsoft.AspNetCore.Mvc.Controllers.ControllerBinderDelegateProvider.<>c__DisplayClass0_0.<<CreateBinderDelegate>g__Bind|0>d.MoveNext()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeInnerFilterAsync>g__Awaited|13_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResourceFilter>g__Awaited|25_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeFilterPipelineAsync()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
   at Microsoft.AspNetCore.Builder.RouterMiddleware.Invoke(HttpContext httpContext)
   at Microsoft.Azure.WebJobs.Script.WebHost.Middleware.FunctionInvocationMiddleware.Invoke(HttpContext context) in //src/WebJobs.Script.WebHost/Middleware/FunctionInvocationMiddleware.cs:line 40
   at Datadog.Trace.ClrProfiler.CallTarget.Handlers.Continuations.TaskContinuationGenerator`3.ContinuationAction(Task previousTask, TTarget target, CallTargetState state)
   at Microsoft.Azure.WebJobs.Script.WebHost.Middleware.ExceptionMiddleware.Invoke(HttpContext context) in //src/WebJobs.Script.WebHost/Middleware/ExceptionMiddleware.cs:line 27