"parameters.profile-id" is a string instead of a number for VP9 in RTP Capabilities dictionary

peetonn commented 3 years ago

A C++ client can not produce nor consume VP9 streams due to mediasoup library rejecting RTP capabilities/parameters in transport.produce(...) and router.canConsume(...) calls. The latter returns false while the former throws:

UnsupportedError: unsupported codec [mimeType:video/VP9, payloadType:98]
    at Object.getProducerRtpParametersMapping (/Users/peetonn/Documents/Work/videocore-media-server/node_modules/mediasoup/lib/ortc.js:503:19)
    at WebRtcTransport.produce (/Users/peetonn/Documents/Work/videocore-media-server/node_modules/mediasoup/lib/Transport.js:243:33)
    at Socket.<anonymous> (/Users/peetonn/Documents/Work/videocore-media-server/server.js:151:64)
    at Socket.emit (node:events:379:20)
    at /Users/peetonn/Documents/Work/videocore-media-server/node_modules/socket.io/lib/socket.js:528:12
    at processTicksAndRejections (node:internal/process/task_queues:76:11)

The problem seems to be the parameters.profile-id being a string instead of a number. When dictionaries are printed out side-by-side for JS and C++ clients, this becomes evident:

JS-based client canConsume rtpCapabilities:

{
  "codecs": [
    {
      "mimeType": "audio/opus",
      "kind": "audio",
      "preferredPayloadType": 100,
      "clockRate": 48000,
      "channels": 2,
      "parameters": {
        "minptime": 10,
        "useinbandfec": 1
      },
      "rtcpFeedback": [
        {
          "type": "transport-cc",
          "parameter": ""
        }
      ]
    },
    {
      "mimeType": "video/VP9",
      "kind": "video",
      "preferredPayloadType": 101,
      "clockRate": 90000,
      "parameters": {
        "profile-id": 0 // <-- a number
      },
...

C++-based client canConsume rtpCapabilities`:

{
 "codecs": [
   {
     "channels": 2,
     "clockRate": 48000,
     "kind": "audio",
     "mimeType": "audio/opus",
     "parameters": {
       "minptime": 10,
       "useinbandfec": 1
     },
     "preferredPayloadType": 100,
     "rtcpFeedback": [
       {
         "parameter": "",
         "type": "transport-cc"
       }
     ]
   },
   {
     "clockRate": 90000,
     "kind": "video",
     "mimeType": "video/VP9",
     "parameters": {
       "profile-id": "0" // <-- a string
     },
...

Original issue raised here -- https://mediasoup.discourse.group/t/failed-to-produce-or-consume-vp9-in-c-native-client-unsupported-codec-vp9-payloadtype-98/2673/3

ibc commented 3 years ago

Problem is that, in libmediasoupclient, we are parsing profile-id as a string rather than as a number. We clearly specify that it must be a number and that's what mediasoup expects to match it.

@jmillan, when we generate RtpParameters and RtpCapabilities in libmediasoupclient, are we "fixing" some of the native obtained fields to convert them from string to number, etc?

ibc commented 3 years ago

@peetonn can you try this branch, please?

versatica / libmediasoupclient

"parameters.profile-id" is a string instead of a number for VP9 in RTP Capabilities dictionary #115