Do we need an L1T1 scalability mode?

w3c / webrtc-svc

W3C Scalable Video Coding (SVC) Extension for WebRTC

https://w3c.github.io/webrtc-svc/

Other

39 stars 14 forks source link

Do we need an L1T1 scalability mode? #48

Closed alvestrand closed 2 years ago

alvestrand commented 2 years ago

Present-day setups will frequently select a scalability mode different from "one spatial layer, one temporal layer" when left to their own devices - the mode is chosen based on the codec and the conditions surrounding the use.

If we want to ensure that the user can strictly control the scalability mode used - do we need to define "L1T1" as a scalability mode with no layers at all?

aboba commented 2 years ago

In WebCodecs, the scalabilityMode is typically omitted if there are no layers. This is awkward if the mode is being logged or considered an integral part of the configutation. So defining L1T1 may make some sense.

fippo commented 2 years ago

the difference between a L1T1 scalability mode being set and none set would be that for VP8 and VP9 would not try to optimize away TL0PICIDX and TID (for VP8) and TID/SID (for VP9)?

That could be useful since it becomes more consistent for SFUs which can always look for those fields and don't need to deal with corner cases where they're not supported.

alvestrand commented 2 years ago

Currently the default mode in WebRTC / VP8 is L1T2. So leaving out the mode will not result in L1T1. (I'm not sure if anyone gets any benefit from L1T1. It might be slightly smaller than L1T2, but would require keyframes at every lost packet, instead of every other lost packet (that can't be repaired). This may matter (or may not).

aboba commented 2 years ago

I have submitted a PR to add the "L1T1" scalability mode value to the table. However, adding the entry would prompt the following behavior questions.

Should "L1T1" be returned in by RTCRtpReceiver/RTCRtpSender.getCapabilities()?
Should "L1T1" be considered a valid scalability mode for use in setParameters() (e.g. as a way to turn off scalable video coding)? Currently this can be accomplished by not including a scalabilityMode member. Does using "L1T1" instead provide value?
Should "L1T1" be returned by getParameters()? Current WebRTC-SVC implementations will not return a scalabilityMode value if scalable video coding is not configured. Neither will an implementation that doesn't support WebRTC-SVC. Returning "L1T1" could help distinguish between those cases - but this can also be accomplished by calling getCapabilities().

alvestrand commented 2 years ago

My thoughts.....

1 - Yes, it should be returned in scalabilityModes. It resolves the ambiguity between [] as "I support everything" and [] as "I support nothing" - in this case, [L1T1] means "I support nothing, but understand what you asked, whereas [] means "I support everything", and <member not present> means "I don't understand scalability". (I'm not sure it is friendly to distinguish between empty list and member-not-present, though....)

2 - Yes, it should be a valid scalability mode in setParameters. When you don't set scalability mode in setParameters, Chrome will currently do L1T2 for VP8, I believe.

3 - Yes, this distinguishes between "I don't understand scalability modes" and "I understand scalability modes, but scalability is off".

aboba commented 2 years ago

Is an argument for returning "L1T1" in RTCRtpReceiver.getCapabilities().
Is an argument for returning "L1T1" in RTCRtpSender.getCapabilities() and also allowing it in RTCRtpSender.setParameters()

aboba commented 2 years ago

It would appear that this issue is addressed by merged PR https://github.com/w3c/webrtc-svc/pull/51

If not, please reopen.