Azure / azure-functions-dotnet-worker

Azure Functions out-of-process .NET language worker
MIT License
403 stars 166 forks source link

Request Cancellation results in ObjectDisposed Exception for Object name: HttpContext #2508

Open dkapadia-jb opened 1 month ago

dkapadia-jb commented 1 month ago

I am running an Azure Functions HttpTrigger application on .NET 8 runtime using Isolated worker model.

When a request is cancelled by a client I am trying to handle this using cancellation token within the function which works fine but before returning a response it throws an exception as below. It seems that the worker is trying to access HttpContext at the end of the request which is already disposed and causes this exception resulting in a function failure when attempting to gracefully handle the OperationCanceledException.

The source code line is referenced below as well. Please help triage this issue.

Potentially linked to #2417

Result: Function 'MyHttpTriggerFunction', Invocation id 'e255c14c-0722-496e-9e4b-c978e1153c42': An exception was thrown by the invocation. Exception: System.ObjectDisposedException: Request has finished and HttpContext disposed. Object name: 'HttpContext'. at Microsoft.AspNetCore.Http.DefaultHttpContext.ThrowContextDisposed() at Microsoft.AspNetCore.Http.DefaultHttpContext.get_Features() at Microsoft.AspNetCore.Routing.RoutingHttpContextExtensions.GetRouteData(HttpContext httpContext) at Microsoft.Azure.Functions.Worker.Extensions.Http.AspNetCore.FunctionsHttpProxyingMiddleware.TryHandleHttpResult(Object result, FunctionContext context, HttpContext httpContext, Boolean isInvocationResult) in D:\a\_work\1\s\extensions\Worker.Extensions.Http.AspNetCore\src\FunctionsMiddleware\FunctionsHttpProxyingMiddleware.cs:line 72 at Microsoft.Azure.Functions.Worker.Extensions.Http.AspNetCore.FunctionsHttpProxyingMiddleware.Invoke(FunctionContext context, FunctionExecutionDelegate next) in D:\a\_work\1\s\extensions\Worker.Extensions.Http.AspNetCore\src\FunctionsMiddleware\FunctionsHttpProxyingMiddleware.cs:line 54 at Microsoft.Azure.Functions.Worker.FunctionsApplication.InvokeFunctionAsync(FunctionContext context) in D:\a\_work\1\s\src\DotNetWorker.Core\FunctionsApplication.cs:line 89 Stack: at Microsoft.AspNetCore.Http.DefaultHttpContext.ThrowContextDisposed() at Microsoft.AspNetCore.Http.DefaultHttpContext.get_Features() at Microsoft.AspNetCore.Routing.RoutingHttpContextExtensions.GetRouteData(HttpContext httpContext) at Microsoft.Azure.Functions.Worker.Extensions.Http.AspNetCore.FunctionsHttpProxyingMiddleware.TryHandleHttpResult(Object result, FunctionContext context, HttpContext httpContext, Boolean isInvocationResult) in D:\a\_work\1\s\extensions\Worker.Extensions.Http.AspNetCore\src\FunctionsMiddleware\FunctionsHttpProxyingMiddleware.cs:line 72 at Microsoft.Azure.Functions.Worker.Extensions.Http.AspNetCore.FunctionsHttpProxyingMiddleware.Invoke(FunctionContext context, FunctionExecutionDelegate next) in D:\a\_work\1\s\extensions\Worker.Extensions.Http.AspNetCore\src\FunctionsMiddleware\FunctionsHttpProxyingMiddleware.cs:line 54 at Microsoft.Azure.Functions.Worker.FunctionsApplication.InvokeFunctionAsync(FunctionContext context) in D:\a\_work\1\s\src\DotNetWorker.Core\FunctionsApplication.cs:line 89. Executed 'MyHttpTriggerFunction' (Failed, Id=e255c14c-0722-496e-9e4b-c978e1153c42, Duration=6668ms) System.Private.CoreLib: Exception while executing function: Functions.MyHttpTriggerFunction. Microsoft.Azure.WebJobs.Script.Grpc: Failed to proxy request with ForwarderError: RequestCanceled. System.Net.Http: The operation was canceled. System.Net.Sockets: Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request.. The I/O operation has been aborted because of either a thread exit or an application request.

https://github.com/Azure/azure-functions-dotnet-worker/blob/dd8398ee4b5b5654cb256ac641e9f8624d86f8fc/extensions/Worker.Extensions.Http.AspNetCore/src/FunctionsMiddleware/FunctionsHttpProxyingMiddleware.cs#L72

jviau commented 1 month ago

Can you share the version of AspNetCore Worker extension package you are using?

dkapadia-jb commented 1 month ago

Hey @jviau, I was initially using v1.2.1 then upgraded to v1.3.1 and still noticed same behavior,

dkapadia-jb commented 4 weeks ago

Just as an update on this, after removing this package, I noticed the worker no longer throws an exception and request cancellation is handled gracefully. So definitely there is a bug in the package even in the latest stable version 1.3.1 Microsoft.Azure.Functions.Worker.Extensions.Http.AspNetCore

maciej-sielski commented 4 weeks ago

as @dkapadia-jb stated I removed this dependency, had to migrate from returning IActionResult to HttpResponseData as stated in https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-http-webhook-trigger?tabs=python-v2%2Cisolated-process%2Cnodejs-v4%2Cfunctionsv2&pivots=programming-language-csharp

also in my Program.cs I had to change

new HostBuilder()
    .ConfigureFunctionsWebApplication

into

new HostBuilder()
    .ConfigureFunctionsWorkerDefaults
dkapadia-jb commented 3 weeks ago

Note: This issue still persists with latest release v.1.3.2

Rihab8 commented 4 days ago

https://github.com/Azure/azure-functions-dotnet-worker/issues/2417#issuecomment-2199847192