aspnet / AspNetKatana

Microsoft's OWIN implementation, the Katana project
Apache License 2.0
967 stars 333 forks source link

ObjectDisposedException within OwinMiddleWare.Invoke for long-running requests #434

Closed mitchkeith closed 2 years ago

mitchkeith commented 2 years ago

This may be similar to this issue, but I'm not certain it's the same https://github.com/aspnet/AspNetKatana/issues/420

We are seeing an ObjectDisposedException during long-running requests. Our service calls into a number of external services, and there are small number of those external requests that may time out occasionally. Sometimes, we see ObjectDisposedException thrown during that scenario. We are wondering if there is any suggested way to handle this type of scenario. Is there a way to ensure that the HttpListenerObject is not disposed while the request is still active? Should we just catch the exception and check for certain conditions which may imply a downstream request has timed out, and handle it more gracefully?

System.ObjectDisposedException: Cannot access a disposed object. Object name: 'System.Net.HttpListenerResponse'. at System.Net.HttpListenerResponse.CheckDisposed() at System.Net.HttpListenerResponse.set_ContentLength64(Int64 value) at Microsoft.Owin.Host.HttpListener.RequestProcessing.HeadersDictionaryBase.Set(String key, String[] value) at System.Web.Http.Owin.HttpMessageHandlerAdapter.SendResponseMessageAsync(HttpRequestMessage request, HttpResponseMessage response, IOwinResponse owinResponse, CancellationToken cancellationToken) at System.Web.Http.Owin.HttpMessageHandlerAdapter.d__0.MoveNext()

Tratcher commented 2 years ago

This seems related to #430 as well. We realized that client disconnects could cause these ODEs. #432 may have fixed this. That hasn't shipped yet, but you should be able to build the dev branch locally and try it.

mitchkeith commented 2 years ago

This seems related to #430 as well. We realized that client disconnects could cause these ODEs. #432 may have fixed this. That hasn't shipped yet, but you should be able to build the dev branch locally and try it.

Thanks for the reply. At what time, or in what version, do you estimate the fix will be available? And how can we consume the fix, when it's available? I assume we'll just need to update our Owin packages to the latest version?

Tratcher commented 2 years ago

It's scheduled for 4.2.1, we're only waiting for the original reporter to verify the fix. If you could build and verify the fix that would also help.

mitchkeith commented 2 years ago

That makes sense; is there any documentation on how I can build from the development branch into my local application? I can try and see if I can get a local repro of the issue and see if it is resolved when using development branch.

Tratcher commented 2 years ago

https://github.com/aspnet/AspNetKatana#source-code, you can run build.cmd from the repo root. That should produce an artifacts folder with the packages. You can add that folder as a nuget feed in your project to reference the local build.

mitchkeith commented 2 years ago

I've been OOF and otherwise busy working on some higher priority work, but may be able to try this out by the end of the week to see if I can get the local repro and validate the fix. I assume the fix hasn't been validated locally yet and it'd still be useful for me to try and do this?

Tratcher commented 2 years ago

The other customer's validation is ongoing. Any confirmation you can give would also help.

sablanc-msft commented 2 years ago

Hi @Tratcher, do you have an ETA for the new release with this fix?

Tratcher commented 2 years ago

The customers doing the validation never got back to us. I anticipate shipping the release in a few months for other reasons.

Tratcher commented 2 years ago

Good news: rolling builds are now available from our new infrastructure. See https://github.com/aspnet/AspNetKatana/blob/main/README.md#signed-rolling-builds

Please try out the new packages, we're hoping to ship this release soon.

Tratcher commented 2 years ago

This has now shipped: https://github.com/aspnet/AspNetKatana/releases/tag/v4.2.1