kesterl An existing connection was forcibly closed by the remote host

[amirjalali1]

asp.net core 2.1

System.IO.IOException: An existing connection was forcibly closed by the remote host ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.Internal.SocketAwaitable.GetResult() at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.Internal.SocketConnection.ProcessSends() at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.Internal.SocketConnection.DoSend() --- End of inner exception stack trace --- at System.IO.Pipelines.PipeCompletion.ThrowLatchedException() at System.IO.Pipelines.Pipe.GetFlushResult(FlushResult& result) at System.IO.Pipelines.Pipe.FlushAsync(CancellationToken cancellationToken) at System.IO.Pipelines.Pipe.DefaultPipeWriter.FlushAsync(CancellationToken cancellationToken) at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1OutputProducer.WriteAsync(ReadOnlySpan1 buffer, CancellationToken cancellationToken) at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1OutputProducer.FlushAsync(CancellationToken cancellationToken) at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1OutputProducer.WriteAsync[T](Func3 callback, T state) at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.WriteChunkedAsync(ReadOnlyMemory1 data, CancellationToken cancellationToken) at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.WriteAsync(ReadOnlyMemory1 data, CancellationToken cancellationToken) at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpResponseStream.WriteAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken) at System.IO.Stream.WriteAsync(Byte[] buffer, Int32 offset, Int32 count) at Microsoft.AspNetCore.WebUtilities.HttpResponseStreamWriter.FlushInternalAsync(Boolean flushEncoder) at Microsoft.AspNetCore.WebUtilities.HttpResponseStreamWriter.WriteAsync(String value) at Microsoft.AspNetCore.Mvc.ViewFeatures.Internal.PagedBufferedTextWriter.WriteAsync(String value) at Microsoft.AspNetCore.Mvc.ViewFeatures.Internal.ViewBuffer.WriteToAsync(TextWriter writer, HtmlEncoder encoder) at Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderLayoutAsync(ViewContext context, ViewBufferTextWriter bodyWriter) at Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderAsync(ViewContext context) at Microsoft.AspNetCore.Mvc.ViewFeatures.ViewExecutor.ExecuteAsync(ViewContext viewContext, String contentType, Nullable1 statusCode) at Microsoft.AspNetCore.Mvc.ViewFeatures.ViewExecutor.ExecuteAsync(ActionContext actionContext, IView view, ViewDataDictionary viewData, ITempDataDictionary tempData, String contentType, Nullable1 statusCode) at Microsoft.AspNetCore.Mvc.ViewFeatures.ViewResultExecutor.ExecuteAsync(ActionContext context, ViewResult result) at Microsoft.AspNetCore.Mvc.ViewResult.ExecuteResultAsync(ActionContext context) at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeResultAsync(IActionResult result) at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeNextResultFilterAsync[TFilter,TFilterAsync]() at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Rethrow(ResultExecutedContext context) at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.ResultNext[TFilter,TFilterAsync](State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeResultFilters() at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeNextResourceFilter() at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Rethrow(ResourceExecutedContext context) at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeFilterPipelineAsync() at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeAsync() at Microsoft.AspNetCore.Builder.RouterMiddleware.Invoke(HttpContext httpContext) at x.xi.Startup.<>c.<b__7_0>d.MoveNext() in x:\x\Startup.cs:line 357 --- End of stack trace from previous location where exception was thrown --- at NWebsec.AspNetCore.Middleware.Middleware.MiddlewareBase.Invoke(HttpContext context) at NWebsec.AspNetCore.Middleware.Middleware.MiddlewareBase.Invoke(HttpContext context) at NWebsec.AspNetCore.Middleware.Middleware.MiddlewareBase.Invoke(HttpContext context) at Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Session.SessionMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Session.SessionMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Builder.RouterMiddleware.Invoke(HttpContext httpContext) at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Diagnostics.StatusCodePagesMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Server.IISIntegration.IISMiddleware.Invoke(HttpContext httpContext) at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)

line 357 is :

            app.Use(async (context, next) =>
            {
                if (context.Request.Scheme == "http")
                {
                    context.Request.Scheme = "https";
                }
                await next.Invoke();  =====>   **this is line 357**
            });

my app is heavily dependent on redis and rabbitmq after this error i got lots of rabbitmq error

RabbitMQ.Client.Exceptions.BrokerUnreachableException: None of the specified endpoints were reachable ---> System.IO.IOException: Unable to write data to the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host

[Tratcher]

That's a standard client disconnect error. Do you have any reason to believe that the client hasn't disconnected?

[amirjalali1]

i have several servers behind load balancer and only one of them has got this error and after getting this error continuously and repeatedly the app couldn't get connected to rabbitmq anymore

[halter73]

@1amirjalai The fact that only one server was affected and both your rabbitmq client and Kestrel are getting the exact same "SocketException: An existing connection was forcibly closed by the remote host" indicates that something killed many different sockets on the same server simultaneously.

I agree that it probably wasn't a bunch of unrelated clients that happened to independently yet simultaneously disconnect midstream, but why do you think Kestrel is to blame? Considering that you see the rabbitmq client sockets getting killed the same way, I would think that whatever component caused these failures would have to some control over both the Kestrel and rabbitmq sockets.

[aspnet-hello]

We periodically close 'discussion' issues that have not been updated in a long period of time.

We apologize if this causes any inconvenience. We ask that if you are still encountering an issue, please log a new issue with updated information and we will investigate.