WebRTC Codec selection API

[Orphis]

こんにちは TAG-さん!

I'm requesting a TAG review of the WebRTC Codec selection API.

We want to be able to change codec with RTCRtpSender.setParameters() in order to do the following:

• Allow different codecs on different encodings.
• Make it possible to change codec without re-negotiating.
• Allow specifying both codec and scalabilityMode with a single API call.

Current users of the WebRTC APIs can achieve the similar functionality with multiple calls that are not quite easy to use (renegotiate with a different codec order in the SDP, which is a heavy operation) and are not as efficient (changing the codec and then changing the scalability mode or other parameters may add an extra I-frame that is not needed). This API also allows to have mixed-codec simulcast, which was not possible previously.

• Explainer¹ (minimally containing user needs and example code): WG interim slides
• Specification URL: spec url
• Tests: N/A, WPT tests under development
• User research: [url to public summary/results of research]
• Security and Privacy self-review²: No impact. The proposed API doesn't change or make available any new information.
• GitHub repo (if you prefer feedback filed there): Github repository, but feedback here please!
• Primary contacts (and their relationship to the specification):
  • Florent Castelli (@orphis, Google), author / spec editor
  • Harald Alvestrand (@alvestrand, Google), WebRTC chair
• Organization(s)/project(s) driving the specification: Google
• Key pieces of existing multi-stakeholder review or discussion of this specification: Pull request
• External status/issue trackers for this specification (publicly visible, e.g. Chrome Status): Github repository

Further details:

• [x] I have reviewed the TAG's Web Platform Design Principles
• Relevant time constraints or deadlines:
• The group where the work on this specification is currently being done: WebRTC WG

We'd prefer the TAG provide feedback as:

💬 leave review feedback as a comment in this issue and @-notify [@orphis]

[torgo]

Hi - Can you please put together an explainer for this so we can start a review? See here https://w3ctag.org/explainers/ for our explainer explainer. Thanks! ✨

[torgo]

Hi @Orphis - just a gentle reminder on this one. Can you please send us a (markdown) explainer - see https://w3ctag.org/explainers/ ? I see you've linked to a presentation but it's really not clear from that what the user need is specifically. I just want to make sure we understand that. Can you also please send us some explicit info on multi-stakeholder support (in particular multiple browser engines)?

[torgo]

Hi folks - I'm going to close this one for now. Please feel free to re-open when you're ready for a review.

[Orphis]

@torgo Sorry for the long delay. Here is an explainer for the API changes: https://github.com/Orphis/webrtc-extensions/blob/explainer/explainers/codec-selection.md Could you reopen the issue please?

[torgo]

@Orphis thanks for this updated explainer. It's really good to see that this proposal enjoys support from multiple browsers. Ideally I'd like to see a bit more discussion of user needs – which you may feel boil down to "more seamless video calls" – but it would be nice to have that spelled out.