search

w3c / webrtc-extensions

A repository for experimental additions to the WebRTC API
https://w3c.github.io/webrtc-extensions/
Other
57 stars 19 forks source link

Allow codec-specific encoder configuration #122

Open murillo128 opened 1 year ago

murillo128 commented 1 year ago

Currently the only way of configure the underlying encoding settings is via setParameters and we have been removing attributes from it, so at the end the only the maxBitrate and scaleResolutionDownBy are supported.

There are several issues with the setParameters api, we have this complex get/set mechanism and we need to specify how all the parameters works across all codecs even if they are codec-specific.

The webcodecs api in the other hand is easier to use/understand/define: https://www.w3.org/TR/webcodecs/#dom-audioencoder-configure

The interesting piece is that it allows codec-specific configurations to be defined, for example this one is for OPUS:

https://github.com/w3c/webcodecs/pull/594/files

dictionary OpusEncoderConfig {
  OpusBitstreamFormat format = "opus";
  [EnforceRange] unsigned long frameDuration = 20;
  [EnforceRange] unsigned long complexity;
  [EnforceRange] unsigned long packetlossperc = 0;
  boolean useinbandfec = false;
  boolean usedtx = false;
};

It would be great if we could have the same of fine grained control over the webrtc encoders. Not sure if this could be applicable to setParameters api or if it would be better to introduce a new api so we are closer to the WebCodecs specs and expose a new encoder object with a congigure methods.

transcevier.sender.encoder.configure({usedtx:true})
aboba commented 1 year ago

@murillo128 Agree that it would be great to figure out how to leverage new WebCodecs features. In addition to the Opus encoder settings there are Video encoder settings that might be of interest (e.g. hwAcceleration). We had a discussion at TPAC 2022 that touched on some of these things. I agree that setParameters may be too limited; we also looked at enhancing Encoded Transform so as to allow WebCodecs to interact with the RtpTransport.