murillo128 / codec-agnostic-rtp-payload-format

Codec agnostic RTP payload format
4 stars 6 forks source link

Document which information should be available to SFUs in case audio/video content is encrypted #7

Open youennf opened 3 years ago

youennf commented 3 years ago

Let's document in the draft the properties we identified that would be useful for SFUs. Two SFU use cases seem to be identified:

For video, there is:

  • at which packet SFU can switch
  • resolution (could be generalised to stream 'quality', for instance frame rate, bit rate as identified by video-layers-allocation00)
  • codec specific information like profile_idc. Maybe this is something that could be skipped?

For audio, there is:

In particular, for OPUS, I check the TOC of the opus frame to know the audio frame length and be able to correctly set the timestamp in recording. Also for recording, I was planning to extract in-band fec in case of packet losses and recreate previous frames as the players do not support PLC.

murillo128 commented 3 years ago

Also, for video it is needed to know what recovery mechanisms should be applied in case there are some missing packets for a frame.

(All this info is covered by the dependency descriptor)