Livestreams should handle short outages gracefully

[TheAssassin]

Describe the problem to be solved

PeerTube supports two kinds of live streams: regular and permanent live streams. Both can't handle short outages gracefully. The regular kind won't allow the sender to reconnect, even if the connection was lost for just a few seconds. The permanent one allows one to reconnect, but treats the new connection as a new live stream. When you have PeerTube record and publish stream dumps automatically, every interruption generates a new recording.

Sometimes, connections can be flaky or degrade over time. This can happen when a stream has already started. A viable solution could be to lower the encoder bitrate (which requires the user to interrupt the stream for a few seconds to change the settings and reconnect). PeerTube however doesn't support this behavior very well.

Describe the solution you would like

In my opinion, both kinds of livestreams should allow users to reconnect within some reasonable amount of time (maybe a few minutes, maybe even more, could be up to the user).

I could imagine adding a "end livestream" feature: users need to press a button before PeerTube considers the stream to be over and starts processing the stream dumps etc.

For the permanent kind, this most likely requires introducing another layer of abstraction which might be called an "event".

I am of course aware that the stream dumps may not necessarily be concatenated as-is, i.e., require re-encoding. An alternative to this would be playing them sequentially. This could be a configuration option, too. When re-encoding the parts, a short message might be displayed showing how long the outage was to make it transparent to the user.

To prevent livestreams from never releasing the dumps, a configurable timeout could be added, either as a global or a per-stream setting.

I'm not entirely sure whether the container PeerTube supports it, but ideally PeerTube would not interrupt the playback but ingest some kind of "stream is offline"/"connection lost"/"stream will be back ASAP" still image.

[kontrollanten]

Great idea. As you mention, the question is how PeerTube should be able to know whether it's a lost connection or a controlled end of the stream. Do you know how Youtube, Facebook and other actors solves this?

[TheAssassin]

No idea, sorry. I think YouTube's default behavior is to end the stream, too. As said, I'd rather just combine a timeout (e.g., 30 minutes after the stream lost the connection, PeerTube considers the stream ended) with a button to explicitly do this in the backend.

[Kinuseka]

I believe facebook allows you to reconnect to the live back if you are disconnected for a reason besides clicking the button, then the facebook starts to wait for you to reconnect (on backend). I am pretty sure livestream apps like OBS has some sort of packet that sends a some sort of stop message when you click the stop streaming button, so they can differentiate whether you willingfully stopped the stream or you just lost connection to them, and if the latter happens then platforms just waits for you until you reconnect again.

Facebook does this, and so does youtube. Peertube however just completely closes the stream.

[TheAssassin]

Unfortunately, common tools like ffmpeg (which PeerTube already uses) and GStreamer don't support this out of the box.

There is a third party GStreamer plugin that automates the process of falling back to another source, see https://coaxion.net/blog/2020/07/automatic-retry-on-error-and-fallback-stream-handling-for-gstreamer-sources/.

You could also use tools OBS Studio for this if they provide RTMP sources.

I'm sure you could build a kind of "proxy" next to your PeerTube server that handles the fallback, but it introduces another element in the chain. Also, this will likely cause re-encoding of the stream (which might not be a problem if your "fallback proxy" is local to your network so you can stream in a quasi lossless quality to the proxy first which then encodes the stream to the final format.

I use a similar concept here to work around some incompatibility between the popular ATEM Mini switcher series and PeerTube's live streaming feature.