search

dotnet / aspnetcore

ASP.NET Core is a cross-platform .NET framework for building modern cloud-based web applications on Windows, Mac, or Linux.
https://asp.net
MIT License
35.2k stars 9.95k forks source link

WebSocket.ReceiveAsync and KeepAliveInterval #3128

Closed sepehr1014 closed 5 years ago

sepehr1014 commented 6 years ago

We're using .NET Core 2.1 RC1 and we've set keep-alive interval using app.UseWebSockets(new WebSocketOptions() { KeepAliveInterval = TimeSpan.FromSeconds(10) })

But when the client is disconnected without sending the close frame (because of connection loss or power outage for instance), WebSocket.ReceiveAsync does NOT return even after 10 seconds. My understanding is that if the server is not receiving a pong frame in 10 seconds, the connection should be regarded as dead and ReceiveAsyncshould return an empty result.

BTW this is behind IIS on Windows Server 2016 with the latest updates installed. We've also set IIS ping pong in application host config. (Though as far as I know .NET Core 2.1 does not use libuv anymore and sockets are handled in managed code)

Tratcher commented 6 years ago

@pan-wang @shirhatti

sepehr1014 commented 6 years ago

Any updates on this issue?

Eilon commented 6 years ago

@Tratcher - where would you suspect that the underlying issue is? In ASP.NET Core? Kestrel? WebSockets middleware? IIS? ANCM?

Tratcher commented 6 years ago

ANCM/IIS or CoreFx WebSockets.

Eilon commented 6 years ago

@Tratcher where do you think we should start? If the bug doesn't belong here, I'd like to at least move it to somewhere more likely.

Tratcher commented 6 years ago

By trying to repro with and without ANCM/IIS.

Eilon commented 6 years ago

@jkotalik - could you take a look at this to see where it should go? Thanks!

jkotalik commented 6 years ago

@sepehr1014 just to clarify, does WebSocket.ReceiveAsync ever return? Also, what version of ANCM are you using (specifically dll file version under Windows\System32\inetsrv.

If you have a repro available, please follow up by emailing me at the email listed on my github profile.

sepehr1014 commented 6 years ago

@jkotalik Sorry for my late response. I was using .NET Core 2.1.0 but I followed up extensively with @pan-wang and came to the conclusion that ping functionality is not yet implemented by ASP.NET Core. So we ended up sending an empty message every minute ...

jkotalik commented 6 years ago

ping functionality is not yet implemented by ASP.NET Core

When you say ASP.NET Core, do you mean the websocket middleware or ANCM itself? I'm fairly sure that AspNetCore implements pings.

pan-wang commented 6 years ago

Ping/pong is handled by windows websocket (networking) and winhttp separately and never hit ANCM. In the case of client disconnected and HTTP.sys did not notify ANCM, ANCM will not send the close frame. The back-end thus will not close the websocket connection and wait on the async receive. I could not repro this disconnect (missing client disconnect from HTTP.sys) issue locally. Need a repro app to further investigation. A workaround is to let the back-end sends some ping message periodically as SignalR does.

moshegutman commented 6 years ago

I'm experiencing this issue too with my app hosted in Azure App Service.

moshegutman commented 6 years ago

Is there a workaround for this? @pan-wang

moshegutman commented 6 years ago

@jkotalik I can repro this issue in IISExpress and IIS. The issue does not occur when running as a self-host server.

pan-wang commented 6 years ago

@moshegutman the workaround depends on the nature of the application. Could you please share your repro so that I can take a look? You can reach me at panwang(at)Microsoft(dot)com.

zlatanov commented 6 years ago

@pan-wang I've created a simple repo here: https://github.com/zlatanov/aspnetcore-websockets-experiments. A simple server that only ever expects Pings and replies with Pongs.

If you run it Kestrel, works as expected. If you run it with IIS or IIS Express, doesn't work. Connect a simple websocket client that doesn't do anything but send pings.

andriusbutk commented 5 years ago

Is there any update? We are having issues responding to pings on Azure app service(ASP.NET CORE 2.1) similarly to everyone above. Is there a known workaround for this issue on app service(ASP.NET CORE 2.1)?

florinciubotariu commented 5 years ago

Are there any details when this issue will be addressed/fixed?

florinciubotariu commented 5 years ago

On another PC, where an old version of AspNetCoreModule was installed (version starting with 7.1), this bug would not reproduce. After upgrading the .NET Core Hosting Bundle with the latest version, the bug is reproducible.

Eilon commented 5 years ago

@jkotalik / @pan-wang - any further ideas on this?

Eilon commented 5 years ago

Talked to @jkotalik and he is investigating.

NPSF3000 commented 5 years ago

We've discovered the same bug in our WS prod code. Setting KeepAliveInterval = TimeSpan.FromSeconds(120) completely failed to detect an incorrectly shutdown client. We suspect this has lead to significant performance degradation due to increased load.

ConnorL33t commented 5 years ago

I'm running into this issue as well. Server is an Azure App service instance.

zlatanov commented 5 years ago

@Eilon @jkotalik Can you share any information about the status of this issue?

Eilon commented 5 years ago

@zlatanov we are working on a resolution to this issue - we will provide an update in this issue when we have more to report.

Rakiah commented 5 years ago

Any information about this ? still not working in ASP.NET Core 2.2.2

davidfowl commented 5 years ago

What isn’t working? In 2.2 we added pings from client to server as well. It has nothing to do with the keep alive interval and in fact you shouldn’t configure that separately because we disable it in favor of our own pings

sepehr1014 commented 5 years ago

@davidfowl What do you mean by your "own" pings? Is there any documentation on how ping-pong should be implemented in ASP.NET Core?

davidfowl commented 5 years ago

Ah my bad I was thinking this was a SignalR issue

Rakiah commented 5 years ago

So how would you terminate the connection on failed keepalive basically ?

zlatanov commented 5 years ago

@davidfowl I have created a Unit Test to illustrate the issue. The branch with the test is here - https://github.com/zlatanov/AspNetCore/tree/ws-keep-alive-issue.

The actual test - https://github.com/zlatanov/AspNetCore/blob/f21fcd3f3ab7b0f585385a65701160e6a4ad6ad0/src/Middleware/WebSockets/test/UnitTests/SendReceiveTests.cs#L109.

The NET Core WebSocket client does send a keep alive frame, but the server socket never receives it.

jkotalik commented 5 years ago

Hey all, the issue originally state with websockets and IIS is fixed fixed. Please download the latest versions of 2.1, 2.2, and 1.x: https://dotnet.microsoft.com/download/dotnet-core/2.1, https://dotnet.microsoft.com/download/dotnet-core/2.2, and https://dotnet.microsoft.com/download/dotnet-core/1.1.

jkotalik commented 5 years ago

@zlatanov are you using IIS in that unit test? It doesn't look like it, so you may have a different issue.

jkotalik commented 5 years ago

Closing as the original issue is resolved.