Align spec /w codec direction decision

[henbos]

How to deal with codecs and directionality was discussed at TPAC 2024:

• Minutes
• Slides
• Recording

There was a strong preference for Proposal A, i.e. to "keep it simple": sendonly transceivers can do all send codecs, recvonly transcievers can do all receive codecs, and sendrecv transceivers can only do codecs and profiles that you can BOTH send AND receive with. So if you want to do fancy unidirectional codecs or profiles you use multiple m= sections.

Let this issue track making sure the spec aligns with Proposal A and allow us to merge other codec related issues as fixed by this issue

[henbos]

The TPAC decision made it clear which direction to go, but we never touched on what happens if the transceiver direction makes it such that none of your codec preferences are applicable anymore (e.g. transceiver is sendrecv but preferences only include recvonly codecs). This issue was pointed out in #2939

[henbos]

Adding "Discuss at next meeting" label to reflect we need to discuss error handling as per previous comment

[henbos]

Laundry list:

• Update the normative text here to clarify this is not just "default receive codec preferences" (e.g. can also be used to set send codec preferences on a sendonly transceiver and so on).
• Update algorithm to allow both send codecs and receive codecs (currently it says to throw InvalidModificationError in the sendonly use case).
• The final steps to create an offer and answer's directionality filtering is actually correct already, however we need to clarify/decide what to do when preferences become empty due to filtering.
• The setParameters algorithm is as far as I can tell correct, no action needed.

[henbos]

The spec used to say a preference is a send codec OR receive codec, this was actually changed in an amendment. So I wonder what our implementation does, if it still does both send and receive or if it was updated as per PR, do you know @fippo @alvestrand ?

[henbos]

Filing crbug to align with any spec changes happening here: https://crbug.com/370792782

[henbos]

To be discussed at October Virtual Interim (slides)

[fippo]

I do not think there needs to be more alignment, we are already doing what RFC 3550 says:

• VP9 (Firefox is not supporting more than the default profile=0)
• H264 (Firefox is only supporting CBP so not testable)

As usual we do need tests though.

[henbos]

We still need to align the spec with the decision such that setCodecPreferences does not throw if you set a sendonly codec (e.g. sendonly transceiver use case)

[fippo]

I thought we had been discussing this before and agreed no changes are needed? Do you have a fiddle or test showing your problem?

https://www.rfc-editor.org/rfc/rfc9429#section-4.2.6

It only affects which codecs the implementation indicates that it prefers to receive, via the offer or answer

If you use sCP on a sendonly transceiver it does not matter much in practice but would only be observable in future O/A after changing the direction.

[henbos]

We discussed this at TPAC and there was a preference for Proposal A, see slides and recording. This issue is about doing what we decided to do.

[aboba]

@fippo IETF RTCWEB and AVTCORE WGs discussed RFC 9429 Section 4.2.6 - and the conclusion was that it was being interpretted by WEBRTC WG in a way that was inconsistent with RFC 3264. The decision at TPAC 2024 fixes that.

[henbos]

Yep, and to really spell this out, the following should work:

const sendOnlyTransceiver = pc1.addTransceiver('video', {direction: 'sendonly'});
sendOnlyTransceiver.setCodecPreferences([sendOnlyCodec]);

const recvOnlyTransceiver = pc1.addTransceiver('video', {direction: 'recvonly'});
recvOnlyTransceiver.setCodecPreferences([recvOnlyCodec]);

const sendRecvTransceiver = pc1.addTransceiver('video', {direction: 'sendrecv'});
sendRecvTransceiver.setCodecPreferences([sendRecvCodec]);

await pc1.setLocalDescription();
await pc2.setRemoteDescription(pc1.localDescription);
await pc2.setLocalDescription();  // Notice how pc2 does not need to set any preference of their own
await pc1.setRemoteDescription(pc2.localDescription);

But the following should NOT work:

const sendRecvTransceiver = pc1.addTransceiver('video', {direction: 'sendrecv'});
sendRecvTransceiver.setCodecPreferences([sendOnlyCodec, recvOnlyCodec]);
... negotiate ....

In the next meeting I will discuss how to handle errors (e.g. if the bad example should result in exceptions or if it should be interpreted as "no preference" after filtering). TBD

[fippo]

https://www.rfc-editor.org/rfc/rfc8829.html#section-4.2.6-1 remains clear on what setCodecPreferences is intended for:

It only affects which codecs the implementation indicates that it prefers to receive, via the offer or answer

I keep looking for a sendonly codec that is not covered by special codec rules like level-asymmetry-allowed which covered the weird (and likely buggy) H264 instance we found.

[henbos]

It does not affect it directly, but it affects it indirectly. This JSEP reading is what led to this regression in the first place, we decided to fix this.

[henbos]

Decision for the remaining error handling was Proposal A of this slide. This issue is now ready for PR.

[henbos]

https://www.w3.org/2024/10/15-webrtc-minutes.html

[fippo]

Let me try to clarify this after cappucino and water with hbos:

• In the past sCP required the codec to be in send and recv. This was a problem for receive-only codecs (which exist in the wild)
• sCP currently requires the codec to be in the recv capabilities
• Future sCP will allow the codec to be in send or recv capabilities. This allows a sender to change the codec order on a sendonly transceiver too (since the remote will match by default) which seems useful. Since we lack a sendonly codec this is not testable in WPT currently.
• Minor ergonomics issue: the input to sCP should probably come from RTCRtpTransceiver.getCapabilities(kind) for consistency with how sCP operates on the transceiver?

That part is 👍

I do not see a need to change those rules based on direction so disagree with the second code block of this comment but that might have been overtaken by events?

I question the conclusion to do "Proposal A" of the slide above. The Working Group seems to make decisions based on opinions while lacking tests which, like this fiddle (took five minutes to write... for the second time, I had shown this in February already) which shows that "deployed code" is doing proposal C. Transceivers can get stopped for a number of reasons, applications must check what was negotiated anyway.

[henbos]

You’re able to recognize the the past sendrecv limitation was a problem for your recvonly use case, but you can’t see that the exact same problem exists for the sendonly usecase? Your questioning of Proposal A has downsides (not fixing sendonly) without upside (what would thay be?). Your opposition also requires not caring what the RFCs referenced in the slides say. I stand by the WG decision.

[fippo]

How does Proposal C (m-line is rejected; this has a downside of having a stopped transceiver after SLD), consistent with "incompatible choices result in a stopped transceiver" not fix sendonly? Transceivers get stopped, they are cheap.

See also #2936 (it seems we are going in circles)