Custom Scalability Structures

w3c / webrtc-svc

W3C Scalable Video Coding (SVC) Extension for WebRTC

https://w3c.github.io/webrtc-svc/

Other

39 stars 14 forks source link

Custom Scalability Structures #3

Open aboba opened 5 years ago

aboba commented 5 years ago

What if an application requires a custom scalability structure not supported in the defined scalability modes? For example, what if an application wants a 4:1 scalability ratio rather than 2:1 or 1.5:1?

Codecs such as AV1 support custom scalability structures, so that the encoder can be configured to support scalability structures beyond those defined in scalability modes.

Do we need a way to support scalability structures beyond those defined in scalability modes?

alvestrand commented 5 years ago

The existing scalability structures show clearly that the design space for scalability structures is very large, and probabliy requires something like a turing-complete language to specify all possible permutations. I would think that an application-defined scalability structure would be best done by defining an interface that installed a name and a function to be called to make the send/size decisions for each frame - and that it's something we should experiment with before trying to standardize it.

aboba commented 5 years ago

There are several ways to address the need for extending the resolution ratios that do not involve support for custom scalability structures:

Add scalability modes to the table. For example, modes featuring 4:1 resolution ratios could be added. It is even possible to add a mode where the resolution ratio is not constant, such as a mode to support simulcast encoding at 1080p, 720p and 360p (e.g. "S3T1p").
Allow layers to be disabled. For example, if a mode with 3 spatial layers were selected and the middle layer were to be disabled we would have a 4:1 ratio (for S3T1) or a 3:1 ratio (for S3T1h).

aboba commented 5 years ago

alvestrand commented 5 years ago

In S3T1, doesn't the top layer depend on the middle layer?

jan-ivar commented 4 years ago

TPAC ACTION: aboba to add a paragraph to the table to describe considerations for adding new values to the table of modes.

aboba commented 4 years ago

With the merger of PR 27, this issue has largely been resolved.

aboba commented 3 years ago

Reopening issue due to a request for support of "flexible mode" in WebCodecs.

aboba commented 8 months ago

At TPAC 2024, a proposed encoding API was presented that would enable arbitrary scalability modes in WebCodecs.