search

w3c / webrtc-svc

W3C Scalable Video Coding (SVC) Extension for WebRTC
https://w3c.github.io/webrtc-svc/
Other
39 stars 14 forks source link

how to specify L3T3? #61

Closed fippo closed 2 years ago

fippo commented 2 years ago 
pc.addTransceiver(stream.getVideoTracks()[0], {
  direction: 'sendonly',
  sendEncodings: [
    {rid: 'q', scaleResolutionDownBy: 4.0, scalabilityMode: 'L3T3'}
    {rid: 'h', scaleResolutionDownBy: 2.0, scalabilityMode: 'L3T3'},
    {rid: 'f', scalabilityMode: 'L3T3'},
  ]    
});

seems like an odd way to do it since each spatial layer is defined by a separate sendencoding. One can not specify a single encoding either

pc.addTransceiver(stream.getVideoTracks()[0], {
  direction: 'sendonly',
  sendEncodings: [
    {scalabilityMode: 'L3T3'}  ]    
});

since that looses the ability to control the downscaling factor (which might be tolerable) and the rid.

murillo128 commented 2 years ago

In webrtc svc we only support SRST https://www.w3.org/TR/webrtc-svc/#dfn-srst so the second snipped is the correct one. There should be no need to specify the rid at all (as we don't support MRST). Regarding controlling the downscale factor, I am not sure if it is supported by all of the codecs, or if they are fixed values in the codec spec.

The first code snippet would set up 3 simulcast layers, each one with L3T3, which is something allowed by the spec IIRC:

When sendEncodings is used to request the sending of multiple simulcast streams using addTransceiver(), it is not possible to configure the sending of "S" scalability modes. The browser may only be configured to send simulcast encodings with multiple SSRCs and RIDs, or alternatively, to send all simulcast encodings on a single RTP stream. Attempting to simultaneously utilize both simulcast transport techniques MUST return OperationError in setParameters() or addTransceiver().

But not sure if we should only allow using temporal scalability modes when simulcast is used. For example:

To set up 3 simulcast layers with 3 temporal layers each, the code should be:

pc.addTransceiver(stream.getVideoTracks()[0], {
  direction: 'sendonly',
  sendEncodings: [
    {rid: 'q', scaleResolutionDownBy: 4.0, scalabilityMode: 'L1T3'}
    {rid: 'h', scaleResolutionDownBy: 2.0, scalabilityMode: 'L1T3'},
    {rid: 'f', scalabilityMode: 'L1T3'},
  ]    
});
Orphis commented 2 years ago

The API as it is doesn't allow you to configure the scaling factor between spatial layers besides picking a supported mode that has a 2x or 1.5x factor (h modes).

aboba commented 2 years ago

As @murillo128 says, sending 3 spatial and 3 temporal layers (L3T3) wouldn't require a RID header extension since the layered bitstream would be sent on a single SSRC. As @Orphis notes it is limited to a 2x or 1.5x scale factor.

fippo commented 2 years ago

There should be no need to specify the rid at all

oh true, its a single RTP session!

It might be worth to split examples 2 + 3 into their own subsections.

aboba commented 2 years ago

I have split examples 2 and 3.

aboba commented 2 years ago

I am going to close this issue. If that is no right, please reopen.