w3c / webrtc-encoded-transform

WebRTC Encoded Transform
https://w3c.github.io/webrtc-encoded-transform/
Other
124 stars 27 forks source link

generateKeyFrame algorithm makes wrong assumptions about number of encoders #145

Open fippo opened 2 years ago

fippo commented 2 years ago

https://w3c.github.io/webrtc-encoded-transform/#KeyFrame-algorithms

  1. Gather a list of video encoders, named videoEncoders from encoder, ordered according negotiated RIDs if any.

VP8 in libvpx based encoders use a single encoder to encode multiple spatial layers. In libwebrtc H264 simulcast is done by a triplet of separate encoders but there seem to be encoders such as OpenH264 with built-in support for multiple spatial layers.

  1. [...] videoEncoders is expected to be empty if the corresponding RTCRtpSender is not active

I don't think that is how things behave.

6 [...] If rid is undefined, set rid to the RID value corresponding to videoEncoder.

The video encoder might not have a rid in non-simulcast cases.

Overall the algorithm seems overly specific and should not attempt to venture into such details. Something along the lines of "ask the encoder to generate a keyframe for the given set of rids or all available layers if the set is empty"

youennf commented 2 years ago

The spec uses an abstract representation which does not have to match how UA are implemented. I am opened to readability/clarification changes, but I prefer being specific about the behaviour, for instance: