RTCP Sender Report

[longnguyen2004]

This is still very WIP, missing a few pieces and having some problems that I haven't root caused yet. However it's in a good enough state that other people can test and improve on.

Note: The current implementation timestamps the packet when sendFrame() is called. If there are extra processing steps before it that causes the audio and video streams to go out of sync, this will not fix it.

TODO:

• [x] Allow to change the packet interval
• [x] Handle SSRC changes
• When the SSRC is changed, the total packet count and total octet count should be reset
• Does the SSRC actually change in the middle of a stream?
• [x] Add local SSRC variable to BaseMediaPacketizer
• Since we're using the SSRC in 2 places now, there should be a local SSRC variable in BaseMediaPacketizer to prevent "prop drilling"
• [ ] Troubleshoot "Unknown (-1)" audio codec
• After some amount of time, Discord decided to just stop receiving/decoding audio. The audio codec field in the RTC Debug dialog shows "Unknown (-1)", and all the other stats shows 0
• Doesn't happen with other watchers. Is it client specific? Do client mods affect it? (I'm using Vencord)
• Haven't tested without the RTCP code yet
• [x] ~Figure out the meaning of the remaining 12 bytes~ They are RTCP Source Description packets, so can be left out

[dank074]

Haven't experienced the Unknown (-1)" audio codec bug, even when streaming for prolonged periods of time (for 100+ viewers who didn't report it happening either).

I don't know if the ssrc will ever change mid stream.

Good work so far 👀

[longnguyen2004]

Then it's probably a problem with my client or something else. I managed to catch a failure while stress testing, and asking other people to join and check also didn't reveal anything unusual.

[longnguyen2004]

Note about the send interval:

The RFC specifies that

• The average RTCP bandwidth should not exceed 5% of the total bandwidth
• The reporting interval should not be lower than 5 seconds
• The interval should be randomized to prevent unintended synchronization of reporting

Since implementing them would create unnecessary complexity, and we aren't trying to be standard compliant (we just need it to work), I've came up with estimates for the intervals that would result in ~5 seconds between reports. Feel free to tweak the formula if I'm wrong though.

[longnguyen2004]

This should be good enough to be taken out of draft

For other people testing this, please check for:

• AV sync over long period
• Audio stability over long period (still wondering if the Unknown (-1) bug is due to my laptop, or due to me messing up something)
• (if possible) The ability to recover and resync from network issues
• Jitter & Jitter Buffer (should be stable and not too high, assuming a good Internet connection)

[longnguyen2004]

My results after 10 hours of continuous streaming:

• Jitter buffer stayed below 100ms the entire time, and jitter was staying around 10ms
• There was a brief period of heavy packet loss on my client, and it had no problem recovering sync
• Unknown (-1) never happened, so probably something was wrong in my laptop/OS the last time

Edit: Please hold while I test further

[longnguyen2004]

Turns out whatever problems I have is due to me using RTSP (note to self: always use SRT). Code is fine and is good to go.