Server-Sent Events (SSE) - add Keep-Alive Messages definition

[liran2000]

"...authors can include a comment line (one starting with a ':' character) every 15 seconds or so."

This is helpful for SSE clients. The SSE client can read the keep-alive messages, and detect whether the connection is alive.

Thing is, since the keep-alive messages mechanism is not mandatory, the SSE client can use this condition only if the keep-alive mechanism is used, since when it is not used, it is a valid scenario that no message is received for a long time, even when the stream is alive.

Problem: keep-alive message is not well-defined. Suggested solution - edit: authors can include a comment line (one starting with a ':' character) starting with ":keepalive" every 15 seconds or so.

This way, the SSE client can detect if keep-alive mechanism is used by checking if it received a keep-alive message as defined.

Opened PR for the solution:

[Yay295]

Why do you need to know that a comment is specifically being used to keep the connection alive?

Also you broke the text encoding in your pull request.

spec link: https://html.spec.whatwg.org/multipage/server-sent-events.html#authoring-notes

[liran2000]

Why do you need to know that a comment is specifically being used to keep the connection alive?

Also you broke the text encoding in your pull request.

spec link: https://html.spec.whatwg.org/multipage/server-sent-events.html#authoring-notes

@Yay295 Thanks for addressing. The SSE client can detect if keep-alive mechanism is used by checking if it received a keep-alive message as defined.
Idea is, the SSE client first detects if keep-alive is used at all since the keep-alive mechanism is not mandatory. Then, if it is used, and no message is received during a configured time interval, the SSE client can mark the stream response as not alive, and disconnect/reconnect to it if needed. Thing is, since the keep-alive messages mechanism is not mandatory, the SSE client can use this condition only if the keep-alive mechanism is used, since when it is not used, it is a valid scenario that no message is received for a long time, even when the stream is alive. A note to add, such mechanism is being used by some networking implementations products. for text encoding I noticed, edited and created a new PR.

[Yay295]

So if this keep-alive method is not used, then the connection gets timed out as normal, but if it is used, a different timeout duration is used?

[liran2000]

@Yay295 if this keep-alive mechanism is not used, the client does not explicitly disconnect/reconnect the stream listening at all if there is no message received, since this is expected to not receive any message for a long time when there are no data changes. But if keep-alive mechanism is used, we can benefit from it - we expect at least keep-alive message to be received every interval by the SSE specs recommendation, thus we can use a timeout and disconnect/reconnect the stream listening when no message received by the timeout.

[domenic]

In the current spec, any comment line is a keep-alive message.

You appear to be advising that servers close connections more often, e.g. if they receive :stay-awake instead of :keepalive, they close the connection. That seems bad for the ecosystem as it would cause more users to get their connections closed.

[liran2000]

@domenic I am not advising any behavior change for servers for closing connections.
Idea is that SSE client is expecting to constantly receive keep-alive messages. When after some time, the SSE client stop receiving the keep-alive messages from the stream, the SSE client can identify it as the connection is not alive, and act accordingly, and disconnect/reconnect to the stream.

A note to add, such mechanism is being used by some networking implementations products.

[domenic]

I'll repeat the question then, in a more generic form.

Currently, any line is a keep-alive message. Why would making a smaller set of messages cause keep-alives be useful, as opposed to the current spec?

[liran2000]

@domenic I will try to answer your question with a detailed example and comparison between current spec to the suggested solution.
Let's look at an example messages flow at the SSE client side:

10:00:00 :keepalive
10:00:10 :unrelated comment
10:00:15 :keepalive
10:00:30 :keepalive

10:05:30: no messages received at the SSE client in last few minutes, the stream may not be alive at this point,
and in some scenarios, the SSE client does not know about it, and not receive the data events - 
which is the problem we try to solve.

Now let's see what the SSE client can do to handle the situation at 10:05:30 where no messages received at the SSE client in last few minutes.
Current spec The SSE client cannot assume anything, since it cannot detect whether keepalive mechanism is used at all, since the comments may all be unrelated comments, and it is a valid scenario to not receive any message for a long time when keep-alive mechanism is not used.
Suggested solution The SSE client can detect that keep-alive mechanism is used, since it got messages starts with ':keepalive'. At 10:05:30 the SSE client knows that no messages received in last few minutes, and then it can handle it and disconnect/reconnect to the stream.

[domenic]

The SSE client cannot assume anything, since it cannot detect whether keepalive mechanism is used at all,

This is not true. Per the current spec, the keepalive mechanism is being used. All comments are keepalive messages.

[annevk]

@liran2000 following https://whatwg.org/faq#adding-new-features would help here.

[Tieske]

@liran2000 is right. This is under-specified.

The typical keepalive (assume 30 seconds) would mean that the client would reconnect if not receiving and message for 1.5 x the interval period (45 seconds seconds).

But for the client to do this, the server first needs to inform the client about the keepalive interval.

The proposed solution however doesn't really work imo, since it relies on the client to detect the interval based on 2 messages. If the connection already stalls after the first one, the client will wait forever on a stalled connection.

I think this would be better solved with a response header, eg: Sse-Keep-Alive: 30, which essentially means: "I the server, expect to send you, the client, a message at least every 30 seconds, and if I do not have any event, I'll send a comment"

[Tieske]

@domenic @annevk any comments on my proposal to add a header. If it is deemed a proper solution, I can propose a PR (as alternative to #7572 ).

[domenic]

As I stated, any message will keep the connection alive. If you would like to have your server communicate how often the client should send messages, you can do that using the SSE protocol itself. We don't need to add a header and get the browser or specification involved in the process.

[Tieske]

Isn't the eventsource the one that monitors the connection state and reconnects automatically? So the eventsource instance would need to know this information. Hence the only option is to make this part of the protocol, because the eventsource has no way of interpreting the data send over the event connection.

This is mixing two different abstraction levels if this is to be done over the SSE connection itself.

[domenic]

The EventSource instance doesn't need to know anything from the header. It knows that every time you send a message, it should keep the EventSource connection alive. If the server and client have a more specific protocol in mind, they can communicate that, but the automatic reconnection algorithm is very simple and doesn't need any extra information like "how often should the server send messages". If the server has specific constraints, then it is responsible for working out something with its clients for doing something beyond the spec's automatic reconnection logic.

[Tieske]

for reference, see: https://github.com/whatwg/html/issues/8297#issuecomment-1292586719

[gdamjan]

The EventSource instance doesn't need to know anything from the header. It knows that every time you send a message, it should keep the EventSource connection alive.

What it (the browser really) doesn't know is when it is not getting the keepalive/comments any more, to force a reconnect. The typical scenario is when the browser changes networks and the TCP connection becomes "stuck" ie. the client/browser can't always detect that it's dead and will not receive any more data, not until the TCP times out (and that can last for hours).

[Tieske]

@gdamjan the thing here is not that "it is not broken", because it clearly is. The thing is that for this team SSE is obsolete, and better alternatives are available, so they won't update. See https://github.com/whatwg/html/issues/8297#issuecomment-1292586719

[AyushNautiyalDeveloper]

@Tieske @domenic @gdamjan is this issue of keeping the sse alive fixed? for me it is breaking after 2mins. though the rate of streaming for me is 12 json objects per second consistently I have tried to add the :/n/n still not able to get the data, Though in this case the server connection is not breaking.

[AyushNautiyalDeveloper]

1. My main question is does it even breaks if the data is coming consistently for the 2 mins. timeout period
2. one more thing does sse remains active even after i have set the connection timeout for the server, I did it for express server and even though the timeout (of 20 second) already occurred still the can see the data until the 2mins limited is reached.