search

ethand91 / mediasoup-ios-client

Mediasoup 3 iOS Client
ISC License
131 stars 65 forks source link

SendTransport.Produce error after onConnect is fired #68

Open xplatsolutions opened 4 years ago

xplatsolutions commented 4 years ago

Seems like I am missing something but the error is not clear where is the problem. I am working with protoo-server in backend but connection and data exchange seems good.

Here is the payload from the server: {"response":true,"id":1594900528,"ok":true,"data":{"id":"e69f9b16-75e7-4db9-9789-472b3461914b","iceParameters":{"iceLite":true,"password":"m1noqsetar4jvnugba45fdnirs9skonp","usernameFragment":"p06nxbxkd2by6e5n"},"iceCandidates":[{"foundation":"udpcandidate","ip":"192.168.2.2","port":40327,"priority":1076302079,"protocol":"udp","type":"host"}],"dtlsParameters":{"fingerprints":[{"algorithm":"sha-1","value":"CE:FC:D9:BA:67:C3:8E:82:75:34:37:2A:E9:01:DA:4C:58:7D:8A:3F"},{"algorithm":"sha-224","value":"69:52:97:0E:FE:59:B1:20:6F:5D:D2:ED:DE:8F:EB:65:D5:01:9D:AE:F8:C8:D1:6B:BD:A7:82:10"},{"algorithm":"sha-256","value":"54:0B:F3:10:3B:3B:DB:9D:4A:F6:D9:F3:AD:51:F5:DA:78:FE:EA:36:38:54:FE:92:93:E1:84:A7:03:85:4F:DE"},{"algorithm":"sha-384","value":"50:D1:9E:B4:B7:24:34:FD:1C:EB:CE:DE:CC:CF:2A:67:59:77:B0:59:34:5B:93:30:B4:C6:E5:76:81:A4:B7:99:F2:5D:A9:3F:5F:85:FA:7D:4D:DB:B7:41:CC:CE:C1:38"},{"algorithm":"sha-512","value":"54:88:3B:7F:12:B2:B5:B2:8B:F4:DC:9A:A1:A5:4F:20:96:36:A7:B9:53:4D:8D:DA:58:D0:00:44:96:4F:86:01:F0:E3:65:CA:FC:75:C5:F4:CC:CF:0D:73:85:9A:0F:1B:13:D9:C0:3F:FA:E9:F4:09:EF:08:39:A6:90:F8:B0:A7"}],"role":"auto"},"sctpParameters":{"MIS":1024,"OS":1024,"isDataChannel":true,"maxMessageSize":262144,"port":5000}}}

Debug info from Device.

[ERROR] transport_wrapper::+[TransportWrapper nativeProduce:listener:track:encodings:codecOptions:appData:]() | [json.exception.type_error.302] type must be string, but is null

onProduce will not get a chance to fire.

In SendTransport.Produce I pas encodings = null and codecOptions = "{\n \"videoGoogleStartBitrate\" : 1000\n}"

private func createProducer(track: RTCMediaStreamTrack, codecOptions: String?, encodings: Array<RTCRtpEncodingParameters>?) {
    self.producerHandler = MediasoupSocketProducerHandler()
    self.producerHandler!.delegate = self.producerHandler!

    let kindProducer: Producer = self.sendTransport!.produce(self.producerHandler!.delegate!, track: track, encodings: encodings, codecOptions: codecOptions)
    self.producers[kindProducer.getId()] = kindProducer

    print("createProducer() created id =" + kindProducer.getId() + " kind =" + kindProducer.getKind())
  }
xplatsolutions commented 4 years ago

Do you think this could be the issue; I was able to pass this stage before converting to web socket signaling, although data seems fine. BTW I have updated a couple of days ago from 1.5.0 to 1.5.3, maybe it is introduced;

xplatsolutions commented 4 years ago

@ethand91 I can't seem to find TransportListenerWrapper in source code, trying to see if I can help debugging this.

ethand91 commented 4 years ago

ListenerWrappers are located at: https://github.com/ethand91/mediasoup-ios-client/blob/master/mediasoup-client-ios/include/wrapper/TransportWrapper.h

xplatsolutions commented 4 years ago

Thanks! Can you provide any assistance/documentation on how to attach the source code to a project and debug? Any chance of looking at the issue and reproduce? I'd appreciate your help as I am stuck for 5 days now with this :/

I updated the framework to 1.5.4 today too but the error is the same.

xplatsolutions commented 4 years ago

It seems the issue is happening when calling this https://github.com/ethand91/mediasoup-ios-client/blob/master/mediasoup-client-ios/include/wrapper/TransportWrapper.h#L28.

Tracking down the code this is called and I see the debug results, https://github.com/versatica/libmediasoupclient/blob/v3/src/Handler.cpp#L216 but this https://github.com/versatica/libmediasoupclient/blob/v3/src/Handler.cpp#L297 never called.

This is where the error I assume happens https://github.com/versatica/libmediasoupclient/blob/v3/src/Handler.cpp#L288.

Inaki, author of mediasoup says it shouldn't be a string but an object.

xplatsolutions commented 4 years ago

It looks like after all the messages below I receive the error.

[DEBUG] Handler::Send() | calling pc->SetLocalDescription():
[DEBUG] PeerConnection::OnSignalingChange() | [newState:have-local-offer]

// Connecting the transport on server here so onConnect is fired
websocketDidReceiveMessage {"response":true,"id":1595231957,"ok":true,"data":{"actionEvent":"connectWebRtcTransport"}} 

[DEBUG] PeerConnection::OnIceGatheringChange() | [newState:gathering]
[DEBUG] PeerConnection::OnIceCandidate() | [candidate:candidate:851307166 1 udp 2122260223 192.168.2.4 61436 typ host generation 0 ufrag /rK5 network-id 1 network-cost 10]
[DEBUG] PeerConnection::OnIceCandidate() | [candidate:candidate:3159874914 1 udp 2122194687 169.254.230.86 58099 typ host generation 0 ufrag /rK5 network-id 2 network-cost 10]
[DEBUG] PeerConnection::OnIceCandidate() | [candidate:candidate:3710925147 1 udp 2122131711 2607:fc20:7c58:c851:8fad:ddf4:c547:ddac 58100 typ host generation 0 ufrag /rK5 network-id 9 network-cost 900]
[DEBUG] PeerConnection::OnIceCandidate() | [candidate:candidate:1046478326 1 udp 2122063615 162.162.189.82 62860 typ host generation 0 ufrag /rK5 network-id 8 network-cost 900]
[DEBUG] PeerConnection::OnIceCandidate() | [candidate:candidate:3710925147 1 udp 2122000639 2607:fc20:7c58:c851:8fad:ddf4:c547:ddac 62861 typ host generation 0 ufrag /rK5 network-id 3 network-cost 50]
[DEBUG] PeerConnection::OnIceCandidate() | [candidate:candidate:3710925147 1 udp 2121935103 2607:fc20:7c58:c851:8fad:ddf4:c547:ddac 62862 typ host generation 0 ufrag /rK5 network-id 4 network-cost 50]
[DEBUG] PeerConnection::OnIceCandidate() | [candidate:candidate:3710925147 1 udp 2121869567 2607:fc20:7c58:c851:8fad:ddf4:c547:ddac 62863 typ host generation 0 ufrag /rK5 network-id 5 network-cost 50]

I assume the problem relies here;

I tried with nil for codecOptions and error is the same, is this related to some server response.

Investigating the iceCandidates response from the server I see a key property "type" with value "host".

{
  "response": true,
  "id": 1595236196,
  "ok": true,
  "data": {
    "actionEvent": "createWebRtcTransport",
    "id": "b8a8c27a-2804-4499-962e-9070a314b276",
    "iceParameters": {
      "iceLite": true,
      "password": "i1j06oqh6zw3zbq0buod3x723o9wg0fd",
      "usernameFragment": "1gezxrt8taveymtr"
    },
    "iceCandidates": [
      {
        "foundation": "udpcandidate",
        "ip": "192.168.2.2",
        "port": 44711,
        "priority": 1076302079,
        "protocol": "udp",
        "type": "host"
      }
    ],
    "dtlsParameters": {
      "fingerprints": [
        {
          "algorithm": "sha-1",
          "value": "89:E3:DB:95:EE:2D:9B:A7:18:9A:AB:3C:CE:DC:28:CD:C2:A8:56:B8"
        },
        {
          "algorithm": "sha-224",
          "value": "E4:B5:1A:46:36:87:54:82:6C:C1:F0:A6:F7:1F:DA:7E:68:2D:FE:51:53:EF:2D:25:F4:1A:F4:6E"
        },
        {
          "algorithm": "sha-256",
          "value": "AF:7C:6C:A4:31:65:89:51:8A:53:66:2F:0A:4C:CD:D3:CB:2E:68:DA:97:B7:17:8E:5F:34:7F:41:05:2E:DA:CE"
        },
        {
          "algorithm": "sha-384",
          "value": "D9:7A:77:54:1E:20:29:7D:95:BF:51:16:10:2C:B9:61:14:85:75:E9:08:97:C3:4D:B1:47:A0:EE:5D:CE:15:B1:C1:00:6F:76:C3:23:32:EA:B4:DF:90:F6:72:4C:10:0E"
        },
        {
          "algorithm": "sha-512",
          "value": "AF:1F:C1:29:21:6D:A9:4B:32:50:F3:CE:5C:FF:1A:67:F2:23:00:C8:90:15:E0:5B:27:D7:07:CC:D7:20:7E:05:51:40:0E:76:AA:C1:F2:3E:0D:15:9F:14:89:44:6A:BE:E2:79:5F:33:68:81:F8:45:4D:E6:4E:D7:67:16:13:DC"
        }
      ],
      "role": "auto"
    },
    "sctpParameters": {
      "MIS": 1024,
      "OS": 1024,
      "isDataChannel": true,
      "maxMessageSize": 262144,
      "port": 5000
    }
  }
}
xplatsolutions commented 4 years ago

Playing around with the server response JSON and now I am getting the below, might be a bit more helpful;

[ERROR] ortc::validateIceParameters() | throwing MediaSoupClientTypeError: params is not an object
[ERROR] device_wrapper::+[DeviceWrapper nativeCreateSendTransport:listener:id:iceParameters:iceCandidates:dtlsParameters:sctpParameters:options:appData:]() | params is not an object
xplatsolutions commented 4 years ago

Repost from mediasoup forum to keep track. Here is the iOS code to create the Device Transport.

print("Server Response - iceParameters:\n\(iceParameters.description)")
print("Server Response - iceCandidates:\n\(iceCandidatesArray.description)")

self.sendTransport = mediasoupDevice!.createSendTransport(self.sendTransportHandler!.delegate!, id: id, iceParameters: iceParameters.description, iceCandidates: iceCandidatesArray.description, dtlsParameters: dtlsParameters.description, sctpParameters: sctpParameters.description, options: nil, appData: nil)

print("Local Transport\nID: \(self.sendTransport!.getId()!)\nConn State: \(self.sendTransport!.getConnectionState()!)\nIsClosed: \(self.sendTransport!.isClosed())")
Server Response - iceParameters:
{
"password" : "4mfx7todqf66mhfq1vo68i1qr8r7va8z",
"usernameFragment" : "b52s6c3s5et910ad",
"iceLite" : true
}

Server Response - iceCandidates:
[
  {
    "protocol" : "udp",
    "foundation" : "udpcandidate",
    "priority" : 1076302079,
    "type" : "host",
    "port" : 45567,
    "ip" : "192.168.2.2"
  }
]
Local Transport:
ID: 3d8bf0a4-07b5-4ccb-b310-054216a848d1
Conn State: new
IsClosed: false
[ERROR] transport_wrapper::+[TransportWrapper nativeProduce:listener:track:encodings:codecOptions:appData:]() | [json.exception.type_error.302] type must be string, but is null

2020-07-20 05:50:35.897595-0400 Phone[4428:2028450] *** Terminating app due to uncaught exception 'RuntimeException', reason: '[json.exception.type_error.302] type must be string, but is null'

First throw call stack:

(0x1bfad5794 0x1bf7f7bcc 0x1052e2ef0 0x1041062c0 0x104105b90 0x10410560c 0x104103610 0x104102e80 0x104107d54 0x10410841c 0x1040ddb9c 0x1040dc390 0x1040db608 0x1040df940 0x10435845c 0x10432b964 0x104a42338 0x104a43730 0x104a45e88 0x104a54d74 0x104a55698 0x1bf7ecb38 0x1bf7ef740)

libc++abi.dylib: terminating with uncaught exception of type NSException
tanaymondal commented 4 years ago

I am also working with proto-server in the backend, getting the same error.

transport_wrapper::+[TransportWrapper nativeProduce:listener:track:encodings:codecOptions:appData:]() | The associated promise has been destructed prior to the associated state becoming ready.

Did you find any solution?