Server Side Event - Socket is not released. [SPR-14819]

[spring-projects-issues]

Harshal Vora opened SPR-14819 and commented

While using SSE Emitter, consider a scenario where a) Client & Server are in a different LAN. Client is behind a NAT, server can be on a public IP or behind a NAT. b) Server is deployed on Tomcat7 c) Timeout is set to infinite (i.e. never times out)

If the client kills the connection, the socket on the server goes into Close_Wait state. This socket is never closed even if you try to send something via sseEmitter until and unless 1) Shut down / Restart the tomcat server 2) Stop this servlet only, not the entire tomcat.

In the second case, connection goes into LAST_ACK state and then gets closed. SSE thread gets killed but sseComplete method is not called.

This behaviour is not seen if both the client and server are part of the same LAN.

The difference that we have observed at the TCP level is that there is no Reset packet when the client and the server are on a different LAN whereas a Reset packet is received by the server if both of them are within the same LAN which kills the connection on the server side and thus sseEmitter thread.

We have also done some testing with Netcat utility and a minimal Java Socket program and they always close the socket on the server side if the client dies.

Currently we have to kill the server side thread on timeout, but this means that the thread is running till timeout is reached (timeout is calculated from the time the connection is created and not from the time the client dies). This has two disadvantages 1) If timeout is too large, then the thread keeps running for long. 2) If the timeout is too small, then the thread might timeout and kill the connection while some transfer is taking place.

Is there any solution for this?

Kindly Note: We are using "org.springframework:spring-webmvc:4.2.1.RELEASE" jar. It does not give me an option to select 4.2.1.RELEASE in the "Affects Version/s" section.

---

No further details from SPR-14819

[spring-projects-issues]

Juergen Hoeller commented

The reason why you can't select 4.2.1 as "Affects Version" is that we're generally only accepting reports against recent Spring Framework maintenance releases. In your case, it'd be great if you could double-check against 4.2.8, just in case the behavior happens to be different there.

[spring-projects-issues]

Harshal Vora commented

I will do that and post it here in the next couple of days.

[spring-projects-issues]

Rossen Stoyanchev commented

This is probably related to the fact that the Servlet async request support does not provide a notification when a connection is closed from the other side.

When using Spring's WebSocket messaging support which includes SockJS fallbacks to HTTP-based transports (including SSE) we send a periodic server-to-client heartbeat to ensure proxies do not think the connection is hung. Furthermore when using STOMP over WebSocket, there are bi-directional heartbeats every 10 seconds by default. Typically when a connection is closed from one side, a heartbeat will fail to write at which point we close the connection pro-actively. So that mitigates the problem.

You could try sending empty data periodically if no other data has been sent. Other than that you can consider Spring's WebSocket messaging support which provides more options and supports a wider range of browsers.

[spring-projects-issues]

Harshal Vora commented

Hi Rossen,

I understand that there is no way to determine when the client goes down as there is no notification. For this purpose we do send continuous heartbeat from the server to the client at the application layer and expect the server to fail when it cannot send data to the client and thus call the onComplete method provided by sseEmitter.

As mentioned earlier, when both client and server are within the same LAN (for ex. both on my office network OR both on my production network) and if the client is killed, server will fail while sending the next heartbeat.

But when client and server are on different LAN's (for ex. my home network and office test network OR my office network and my production network) and if the client is killed, then the server keeps sending these heartbeats and never fails. SSE thread keeps running.

Since we have tested this across combination of different networks we are fairly confident that the difference in behaviour has nothing to do with the firewall or any other network settings of the environment.

The only difference that we have observed at the TCP level is that there is no Reset packet when the client and the server are on a different LAN

[spring-projects-issues]

Rossen Stoyanchev commented

hi sorry for the slow response. Thanks for the extra detail.

The one difference with STOMP over WebSocket is that the heartbeat is bi-directional in which case the server will close the connection if the client does not send a heartbeat. When running with SockJS fallbacks, for HTTP streaming or polling (including an EvenSource-based transport) naturally the client needs to send heartbeats in a separate connection over HTTP POST. That's all done transparently for the application which always uses a WebSocket API even when HTTP fallbacks may be used underneath.

Short of doing something similar in your application with the client also sending periodic heartbeats to the server, I am not sure what else can be done.

In case there are more network-level fixes to the problem, this might be a better question for the Tomcat mailing list.