Auto adjust video quality to maintain fluid video playback

[ageekhere]

Hi, is there a way we can adjust the video quality to maintain fluid video playback over a given connection? It would be good if it can do it automatically. So on slow connections the video bitrate would be lowered until fluid video playback is achieved.

[superdump]

There are pull requests under review that add the SCReAM congestion control algorithm. This algorithm identifies congestion in the network and adjusts the bitrate accordingly.

The signalling and feedback formats are being standardised so it will only be usable between two OpenWebRTC clients currently. Later, when those parts are standardised and Google have implemented their own algorithm in the webrtc.org stack, we should be able to have interoperability with Chrome and Firefox, with very good bitrate adaptation.

[superdump]

A required GStreamer plugin for SCReAM: https://github.com/EricssonResearch/openwebrtc-gst-plugins/pull/43

The OpenWebRTC parts of SCReAM: https://github.com/EricssonResearch/openwebrtc/pull/413

[ageekhere]

“add the SCReAM congestion control algorithm” Would this be added to the web server end “channel_server.js” or the client end Bowser app?

Would this also take into account max CPU usage for a client?

Example Client 1 has a 1920x1080p Webcam. Client 2 has a low end smart phone or tablet. Lan connection.

In this example the data rate is not the bottleneck but rather the processing power of client 2, would the video quality be able to downgrade to a level in which client 2 would be able to play the video from client 1?

[superdump]

It would be part of OpenWebRTC (or any other WebRTC stack that cares to implement the algorithm) and it communicates necessary information directly to other clients. Nothing to do with the web server.

There are other mechanisms for clients to specify their capabilities - this is also part of the WebRTC client software stack. I believe it is now possible to signal (via SDP) the maximum supported resolution and frame rate, as well as other things. This should be implemented and exposed to applications I think.

[ageekhere]

Sorry I am not understanding, is SCReAM done on the client side or server side? If client 1 is Firefox and client 2 is Chrome and they connect to a private webserver (channel_server.js). If the clients are communicating directly and SCReAM is not done by the web server but by the clients how is this going to be implemented? I can understand that if both clients are using Bowser than that would work, but what about other web browsers?

[superdump]

The current bitrate adaptation algorithms all work in the clients.

The web server is only used by an application as a known point to which clients can connect in order to send information to each other about how to connect to each other directly and what data each other support for audio, video formats and so on. This is called signalling.

As part of the signalling, each client will express that it supports certain types of feedback or perhaps that it uses some bitrate adaptation algorithm. Then when the media is flowing between the two clients peer to peer (not via the signalling server) they can send metadata or feedback to each other about the data flow of the streams so that they can figure out how to adapt and adjust the encoder bitrate or use other adaptation mechanisms.

SCReAM is one algorithm that works by the receiving client sending feedback about whether packets have been received or not. If not, this is most often due to congestion (buffering due to inability to send the data through) in the network. A strategy then is to reduce the bitrate in order to try to get all of the data through.

The Google congestion control (bitrate adaptation) algorithm, that is the current one being used by Firefox and Chrome, adds a send time header to the sent RTP packets so that the receiver can work out how long the packet took to get through the network. When it sees any delay, it observes that as congestion and sends a feedback message to the sender requesting that it send with a specific lower bitrate that it thinks could work for the channel. It keeps observing and adjusts the bitrate up or down as necessary.

SCReAM will initially only be implemented in OpenWebRTC so initially it will only work between OWR clients. Google has a new algorithm that works similarly to SCReAM and multiple parties including Google and Ericsson are working on standardizing the signalling part and the feedback part so that even if the clients use different algorithms, as long as they support the signaling and feedback, they can be interoperable.

[marcin-sielski]

Thanks for very thorough summary. Could you please give an update on progress of the standardization process?

[superdump]

I don't know about the standardization part but that shouldn't require much adjustment once it is ready.

The algorithm, elements and integration should be landing soon with support for OWR <-> OWR calls.

[ageekhere]

What is the status of SCReAM?

[superdump]

SCReAM is merged. Standardisation of feedback and as SDP signalling ongoing.

[stefanalund]

Read more here http://www.openwebrtc.org/blog/2015/11/19/your-openwebrtc-calls-are-now-congestion-controlled

[ageekhere]

Does SCReAM work the same with: Chrome calling bowser (tried a test call over the internet and the video was laggy, fine on lan). Firefox calling bowser (UPDATE tried a test call over the internet and the video was laggy, fine on lan). IE calling bowser (not yet tested). Bowser calling Bowser (UPDATE video was exceeding available bandwidth).

Ok I think there is an issue with SCReAM that it is not lowering the video quality enough to maintain fluid video playback on a 10mbps down 1mbps up connection.

If someone can confirm this then this should be opened as a bug.