Idea: Adaptive bitrate based on queue fill factor

[Roenbaeck]

I noticed that you have a queue in BaseSender, which is good. However, it's hard coded to 200. If we take streaming to YouTube as an example, you can set some different latency options for your stream. This basically sets up the queue size on the server side, and if we start a stream on the device nothing will show until the queue is filled on the server. As long as there are no network bandwidth drops, the queue on the client will remain empty and the queue on the server remains full.

If the bandwidth should drop below what is needed to send the target bitrate, the queue on the server will start to empty and the queue on the client will start to fill. If the situation recovers quickly and the bandwidth is again over what is needed for the target bitrate, the client will empty the queue by sending the data as quickly as possible. So in this scenario we recover without ever having to adapt the bitrate. What would be good here is if the queue in the client is about the same size as the queue on the server, since they are working in tandem, and not hard coded to 200.

The other scenario is when the bandwidth drops over a longer period. If we do nothing, the queue on the server will become empty and the queue on the client will become full, resulting in dropped frames. An adaptive bitrate algorithm can use the fill factor of the queue as a trigger to how aggressively it should lower the bitrate. Ideally, this should be done gradually and needs to be tested to find the best thresholds. An example would be to reduce the bitrate to 0.75x the target bitrate when the queue is 25% filled, 0.5x the bitrate if the queue is 50% filled and 0.25x the bitrate if the queue is 75% filled. Other options, that can be combined with bitrate, are to reduce resolution and frame rate.

If the queue on the client and the server have the same size, we can determine when frames will be dropped, and we can do our best to extend the time before the queue becomes full by adapting bitrate, frame rate, and resolution.

[pedroSG94]

Hello,

Currently you can change the queue cache size as you want:

genericCamera.getStreamClient().resizeCache(newSize: 200)

[Roenbaeck]

Thanks, I missed that you could do that. Will that result in a "User Control Message" to the sever to sync the size in the handshake? See: https://rtmp.veriskope.com/docs/spec/#717user-control-message-events

[pedroSG94]

Hello,

No, I'm not using that command. Also, in my library the cache is not in millisecond like the command say. The cache size is in number of packets in my case