S modes and a single active simulcast layer

[Orphis]

Should we allow S modes to work with simulcast layers if there are not more than 1 active layer? At the moment, the spec will reject the configuration if there are more than 1 layer, no matter the state. We could allow S modes on the first layer only if the first layer is active, or none are active.

I think it could be useful to negotiate simulcast all the time for applications in case SVC isn't supported and we fallback to a codec that isn't able to accommodate the SVC configuration, in which case, we may want to configure simulcast.

For example:

pc.addTransceiver(track, {
      direction: 'sendonly',
      sendEncodings: [
        {rid: 'q', scaleResolutionDownBy: 1.0, scalabilityMode: 'S3T3', active: true}
        {rid: 'h', active: false},
        {rid: 'f', active: false},
      ]    
});
// Later, we figure out VP8 was negotiated and scalabilityMode automatically downgraded,
// getParameters() could return something like:
[
        {rid: 'q', scaleResolutionDownBy: 1.0, scalabilityMode: 'L1T3', active: true}
        {rid: 'h', active: false},
        {rid: 'f', active: false},
]
// So we update our configuration with setParameters():
[
        {rid: 'q', scaleResolutionDownBy: 4.0, scalabilityMode: 'L1T3', active: true}
        {rid: 'h', scaleResolutionDownBy: 2.0, scalabilityMode: 'L1T3', active: true},
        {rid: 'f', scaleResolutionDownBy: 1.0, scalabilityMode: 'L1T3', active: true},
]    

This would prevent the need to renegotiate to add a second transceiver to do simulcast or to negotiate 2 transceivers upfront, 1 for SVC and 1 for simulcast fallback.

[fippo]

This would prevent the need to renegotiate to add a second transceiver to do simulcast or to negotiate 2 transceivers upfront, 1 for SVC and 1 for simulcast fallback.

I would prefer separate transceivers for simulcast and SVC. I am not sure we have a way to express "i can do both" in the SDP even?

[aboba]

There are quite a few potential corner conditions:

1. In the example, if AV1 were negotiated, then attempts to set h and f layers active will fail.
2. Interaction of scaleResolutionDownBy with S modes. I guess it would scale all the S mode layers down equally?
3. Sending RIDs with S modes. Could this confuse the SFU? The Offer would show 3 simulcast layers with distinct RIDs (and SSRCs). SFU would not see the mode (because SVC modes aren't negotiated in SDP). So SFU could Answer and accept the 3 layers, then be surprised by receiving S mode instead.

[aboba]

@fippo SVC is not negotiated in SDP (at least with H.264/AVC, VP8, VP9 and AV1). So either SVC signaling is handled out-of-band (e.g. exchange of getCapabilities()) or it is just set in the encoder. So there is no way to express "I can do both". It hasn't been an issue so far because there is not much S mode deployment as far as I know.

[Orphis]

1. Yes, setting h and f layers active would fail, we still don't want actual simulcast and S modes combined.
2. I always think of scaleResolutionDownBy as reducing the resolution before it reaches the encoder which will do more scaling as necessary for the various SVC layers.
3. Maybe. Maybe the information in the various dependency descriptors could indicate the SFU which mode is currently used.

[Orphis]

In general, the goal would be to avoid a renegotiation to add a transceiver with simulcast in advance if needed. If the SVC mode is rejected, we would need to create a new transceiver with simulcast, stop the singlecast one. The transition could be as equally confusing for SFUs.

[aboba]

I have submitted a PR.

[aboba]

@orphis Thinking a bit more about this, it seems that an error need only be thrown when an 'S' mode encoding has active = true and there is also more than one encoding that has active = true. If the 'S' mode encoding has active = false, it shouldn't matter how many encodings have active = true.

[alvestrand]

backing off to the 100.000 foot level, do we have enough justification for preventing people from being stupid on this point (enabling multiple S-mode layers)?

This line of argument seems to make the rules very complex for deciding which configurations are allowed or disallowed, and I don't see a huge argument that the pain resulting from "stupid" configurations being allowed is big enough to warrant making the rules complex.

[aboba]

@alvestrand It's a good question. The restriction arose from a concern that requiring support of mixed simulcast modes would overly complicate implementations, without providing any real value. So the restrictions are there to make the implementer's life easier, not harder. If they are overly complex, we can pare them back or remove them.

[Orphis]

I believe the main concerns initially were on the SFU side to support such configuration. But it's true that it's also a bit complicated for WebRTC implementations to support well.

[aboba]

Closed after merger of PR https://github.com/w3c/webrtc-svc/pull/86

Please reopen if further changes are needed.

[dontcallmedom-bot]

This issue was discussed in WebRTC Feb 2023 Meeting – (Issue #73 🎞︎)