search

w3c / webrtc-pc

WebRTC 1.0 API
https://w3c.github.io/webrtc-pc/
Other
438 stars 115 forks source link

Codec negotiation and level IDs #3020

Open henbos opened 1 week ago

henbos commented 1 week ago

If you are capable of level-id=X then you are also capable of level-id=Y where Y<X, assuming mimeType, profile-id, tier-flag and tx-mode is the same. I.e.

Today's filter seems to assume that every unique combination of codec, profile and level-id is included in the list:

If transceiver.direction is "sendonly" or "sendrecv", exclude any codecs not included in the list of implemented send codecs for kind.

If transceiver.direction is "recvonly" or "sendrecv", exclude any codecs not included in the list of implemented receive codecs for kind.

But in reality, user agents only list the highest level-id they support. Such as 186, rather than all possible values between 0 and 186 as separate codec values. This means you can offer one level-ID from your getCapabilities but this gets downgraded to a level-ID that is not in your getCapabilities with interesting consequences:

henbos commented 1 week ago

@aboba You have pointed this (or something similar) out before when I didn't think about it. Does offering and answering 156 in the above example if direction=sendrecv make sense to you? (If direction is sendonly or recvonly we would still offer/answer the highest value we support in that direction, so this issue only becomes apparent when direction is sendrecv)

henbos commented 1 week ago

I see that the level-id way of thinking is consistent with our take on codec directionality as per https://github.com/ietf-wg-avtcore/draft-ietf-avtcore-hevc-webrtc/pull/29/files and we should just clarify the same applies here.

aboba commented 1 week ago

@henbos Yes. Note that for HEVC the "media format" doesn't include level-id.

henbos commented 1 week ago

What are the implications of "media format" not including level-id? That link says "Implementations SHOULD include this parameter within SDP" but does not mention it in the fmtp sentence?

Today in Chromium if I call RTCRtpSender.getCapabilities/RTCRtpReceiver.getCapabilities with

--enable-features=WebRtcAllowH265Send,WebRtcAllowH265Receive --force-fieldtrials=WebRTC-Video-H26xPacketBuffer/Enabled/

I do see the level-id in the codec.sdpFmtpLine values. In your view, is that a bug or a feature?

henbos commented 4 days ago

On the SDP negotiation side, it is my understanding that we know how to deal with level-ids and unit tests are being added in https://webrtc-review.googlesource.com/c/src/+/367322 (C++ layer, no WPTs yet, we can also do WPTs later).

However on the JS API side I think it might get confusing if setParameters rejects with values from getCapabilities?

taste1981 commented 3 days ago

I would expect setParameters only select codecs as you get from getParameters(). What you get from getParameters() is related to the current direction of transceiver.

RtpSender.getCapabilities() now only returns the supported codecs when the transceiver is in the sendonly direction. It is not reporting capability for a sendrecv transceiver(unless we have a transceiver.getCapability() API).

dontcallmedom-bot commented 2 days ago

This issue had an associated resolution in WebRTC November 19 2024 meeting – 19 November 2024 (Issue #3020: Codec negotiation and level IDs):

RESOLUTION: Proposal A

henbos commented 1 day ago

@taste1981 yes you should only be able to pick from what was negotiated, we don't want to change that. The issue here is specifically about the level-id of a codec getting "downgraded" for your codec/payload type that was in fact accepted, see slides.