Allow having inactive by default codecs

[murillo128]

I am currently adding support for vp9 profile 1 and 3 and h264 Hi10P and Hi422P to libwebrtc, and the SDP generated is HUGE:

Most of these codec/profiles won't be used on a daily base, so I would be great if there could be a way of having them disabled by default so they don't appear on the offer (but they are negotiated on the answer) and have an API to enable them by the app if needed.

I don't think we have anything similar at the moment.

[fippo]

we've run into similar issues before, both when enabling flexfec-03 (recvonly because uhm...) as well as the (recvonly) YUV444 addition that comes to Chrome 101 and this has already resulted in running out of payload types in the upper range (see here for what you have been spared)

setCodecPreferences is great but we have the same problem as with #99.

[murillo128]

Yes, the problem is very similar as the one exposed in #99 for the rtp header extensions, so we could reuse the same proposed solution for the codec APIs.

[murillo128]

how about adding an extension to the transceivers similar to the one for the rtp header extensions?

partial interface RTCRtpTransceiver {
  undefined setOfferedCodecs(sequence<RTCRtpCodecCapability> codecsToOffer);
  readonly attribute FrozenArray<RTCRtpCodecCapability> codecsToOffer;
  readonly attribute FrozenArray<RTCRtpCodecCapability> codecsNegotiated;
};

we would need to add also a property offeredByDefault to the RTCRtpCodecCapability to signal if the codec is included in an initial offer by default or not.

[fippo]

the general problem with a codecsToOffer accessor is that we run into the same issue that we have with RTCRtpSender.getCapabilities(kind) being sync while some of the hardware-enabled codecs might need time to initialize (and I am not a fan of the MC approach).

Maybe we should have an async transceiver.getCapabilities (we can still change the rtp header extension api since it hasn't shipped yet)? Negotiated codecs can be retrieved via getParameters

[aboba]

This issue is on the agenda for the April 26 WEBRTC WG Interim meeting. Slides are here.

[fippo]

:joy: @ slides

for the direction i'd go with sendrecv, sendonly, recvonly and stopped (there are some implications for the transceiver direction). One can then either filter unwanted codecs or set them to "stopped".

[fippo]

how about this (which may also solve #95):

partial interface RTCRtpTransceiver {
  undefined setOfferedCodecs(sequence<RTCRtpCodecCapability> codecsToOffer);
  undefined setOfferedRtpHeaderExtensions(sequence<RTCRtpHeaderExtensionCapability> headerExtensionsToOffer);
  Promise<RTCRtpCapabilities> getCapabilities();
};

getCapabilities can be async method and we don't need the "kind" even. RTCCodecCapability would need to be extended by the direction (which was previously implied by whether it came from sender or receiver). Not sure if we need setOfferedCodecs even, extending setCodecPreferences should be limited to the direction.

Since setOfferedHeaderExtensions hasn't shipped yet I assume we can still make changes to it. While that information is sync it would be nice for symmetry reasons.

Not sure I want the offered + negotiated codecs / extensions, those can be easily extracted from the SDP (as local and remote codecs) so syntactic sugar.

naive polyfill: https://jsfiddle.net/fippo/rcguv56L/8/

[aboba]

Discussed at the April 26 2022 Virtual Interim. Slides Minutes