Glimesh / janus-ftl-plugin

A plugin for the Janus WebRTC gateway to enable relaying of audio/video streams utilizing Mixer's FTL (Faster-Than-Light) protocol.
https://hayden.fyi/posts/2020-08-03-Faster-Than-Light-protocol-engineering-notes.html
GNU Affero General Public License v3.0
44 stars 11 forks source link

🕑 Support playout delay extension #134

Closed danstiner closed 2 years ago

danstiner commented 2 years ago

TODO

Summary

This PR enables support for the experimental playout delay extension described here: https://webrtc.googlesource.com/src/+/refs/heads/main/docs/native-code/rtp-hdrext/playout-delay

The intention is to test playout delay as a workaround for the issue discussed in #101, at least until we can look deeper at changing Chrome to correctly determine a playout delay.

Requirements

Other Notes

Suggest trying a minimum in the range of 200-400 and a maximum of at least 2000, but no more than 10,000.

Expected outcome is that rendering will be delayed by the min value set. You can try setting a large minimum to see the effect (e.g. 5000), or look at chrome://webrtc-internals/ like the following image.

This shows a stream with a min playout of 200ms. The delay jumps up to 250ms at one point, but never drops below the configured 200ms min delay. image

danstiner commented 2 years ago

Testing approach was to run Linux VMs locally and simulate a slow client connection. This takes a couple hours of setup but is pretty straightforward. Note the network device I use is eth1, this may be different for your setup.

Scenario 1

Limit egress traffic to a connection just barely fast enough to handle a 6mbps stream. The 320kbit burst buffer lets the burst of packets on keyframes "back up" and be delivered milliseconds later when there is bandwidth available again, similar to a real rate limited connection. (Note with my test stream there was no packet loss, you may have to tweak your test stream, burst size, and/or rate to achieve similar results).

tc qdisc add dev eth1 root tbf rate 6mbit burst 320kbit latency 200ms

Tested with sample stream I've been using that is mostly stable white noise with a small changing clock. (This leads to tiny p-frames with large i-frames, which is the scenario causing dropped frames we are trying to address with this change).

Outcome with no delay enabled. You can see brief pauses in the clock every two seconds that corresponds with keyframes. (The pauses can be even worse under real conditions, depending on your available bandwidth, type of content, encoder settings, other traffic on your network, etc etc)

https://user-images.githubusercontent.com/52513/145135103-e02f94ac-a58e-4ca6-9cee-e36ff9b4edb6.mp4

Outcome with min delay of 400ms set. There are no pauses. chrome://webrtc-internals/ can also be used to confirm this by looking at number of dropped frames in both outcomes.

https://user-images.githubusercontent.com/52513/145135234-e12eb29d-29e5-49c0-b0d4-2ee41be52297.mp4

To cleanup traffic control rules, either reboot or do:

tc qdisc del dev eth1 root

References: