search

w3c / webrtc-pc

WebRTC 1.0 API
https://w3c.github.io/webrtc-pc/
Other
438 stars 115 forks source link

getCapabilities question #2696

Open fippo opened 3 years ago

fippo commented 3 years ago

I was looking at https://webrtc.github.io/samples/src/content/peerconnection/audio/ which in Chrome 96+ shows audio/RED with a sdpFmtpLine 111/111.

This pulls from RTCRtpSender.getCapabilities("audio").codecs

[
 {
  "channels": 2,
  "clockRate": 48000,
  "mimeType": "audio/opus",
  "sdpFmtpLine": "minptime=10;useinbandfec=1"
 },
 {
  "channels": 2,
  "clockRate": 48000,
  "mimeType": "audio/red",
  "sdpFmtpLine": "111/111"
 },
 {
  "channels": 1,
  "clockRate": 16000,
  "mimeType": "audio/ISAC"
 },
 {
  "channels": 1,
  "clockRate": 32000,
  "mimeType": "audio/ISAC"
 },
 {
  "channels": 1,
  "clockRate": 8000,
  "mimeType": "audio/G722"
 },
 {
  "channels": 1,
  "clockRate": 8000,
  "mimeType": "audio/PCMU"
 },
 {
  "channels": 1,
  "clockRate": 8000,
  "mimeType": "audio/PCMA"
 },
 {
  "channels": 1,
  "clockRate": 32000,
  "mimeType": "audio/CN"
 },
 {
  "channels": 1,
  "clockRate": 16000,
  "mimeType": "audio/CN"
 },
 {
  "channels": 1,
  "clockRate": 8000,
  "mimeType": "audio/CN"
 },
 {
  "channels": 1,
  "clockRate": 48000,
  "mimeType": "audio/telephone-event"
 },
 {
  "channels": 1,
  "clockRate": 32000,
  "mimeType": "audio/telephone-event"
 },
 {
  "channels": 1,
  "clockRate": 16000,
  "mimeType": "audio/telephone-event"
 },
 {
  "channels": 1,
  "clockRate": 8000,
  "mimeType": "audio/telephone-event"
 }
]

Showing the payload type doesn't make much sense at that point since it does not relate to anything that is currently there.

We don't do that for video/rtx where only a single rtx codec with no apt shows up:

[
 {
  "clockRate": 90000,
  "mimeType": "video/VP8"
 },
 {
  "clockRate": 90000,
  "mimeType": "video/rtx"
 },
 {
  "clockRate": 90000,
  "mimeType": "video/VP9",
  "sdpFmtpLine": "profile-id=0"
 },
 {
  "clockRate": 90000,
  "mimeType": "video/VP9",
  "sdpFmtpLine": "profile-id=2"
 },
 {
  "clockRate": 90000,
  "mimeType": "video/H264",
  "sdpFmtpLine": "level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f"
 },
 {
  "clockRate": 90000,
  "mimeType": "video/H264",
  "sdpFmtpLine": "level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42001f"
 },
 {
  "clockRate": 90000,
  "mimeType": "video/H264",
  "sdpFmtpLine": "level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f"
 },
 {
  "clockRate": 90000,
  "mimeType": "video/H264",
  "sdpFmtpLine": "level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f"
 },
 {
  "clockRate": 90000,
  "mimeType": "video/AV1X"
 },
 {
  "clockRate": 90000,
  "mimeType": "video/red"
 },
 {
  "clockRate": 90000,
  "mimeType": "video/ulpfec"
 }
]

Which does not make much sense either?

Reading http://draft.ortc.org/#dom-rtcrtpcodeccapability do we need to bring preferredPayloadType back?

https://w3c.github.io/webrtc-pc/#rtcrtpcodeccapability seems a bit less specific.

@aboba may have an opinion.

alvestrand commented 2 years ago

would not like to bring back preferredPayloadType. but how do we know that red 111/111 is opus-with-red and not g711-with-red?

fippo commented 2 years ago

oh that one is easy to answer: g711 with red won't come at 48000. But I am worried what happens if satin makes it to a browser...

aboba commented 2 years ago

This is getCapabilities(), so the goal is to indicate what the capabilities are. Agree that 111/111 doesn't make much sense there. Even if we knew what payloadType 111 was, is this trying to say that only that payloadType is supported for RED? If it is possible to use RED with other codecs, then we could just remove the 111/111 from the capabilities.