associated promise error

[MartinP7r]

I again, When I try to handle the SendTransportListener events with a separate class I run into this error again:

[ERROR] transport_wrapper::+[TransportWrapper nativeProduce:listener:track:encodings:codecOptions:appData:]() | The associated promise has been destructed prior to the associated state becoming ready. The associated promise has been destructed prior to the associated state becoming ready.

when calling sendTransport.produce(...

the HandlerClass looks like:

class SendTransportHandler: NSObject, SendTransportListener {

    private let socket: Socket
    weak var delegate: SendTransportListener?

    init(socket: Socket) {
        self.socket = socket
        super.init()
    }

    func onProduce(_ transport: Transport!,
                   kind: String!,
                   rtpParameters: String!,
                   appData: String!,
                   callback: ((String?) -> Void)!) {
        l.info("** onProduce")
        guard let kind = kind else { return }
        let rtp = JSON(parseJSON: rtpParameters)
        let params: JSON = [
            "transportId": transport.getId() ?? "",
            "kind": kind,
            "rtpParameters": rtp
        ]
        guard let obj = params.dictionaryObject else { return }
        socket.request("produce", data: obj) { res in
            l.debug("xx produce result \(res)")
            callback(res["id"].stringValue)
        }
    }

    func onConnect(_ transport: Transport!, dtlsParameters: String!) {
        guard let params = JSON(parseJSON: dtlsParameters).dictionaryObject else { return }
        socket.syncRequest("connectProducerTransport", data: [ "dtlsParameters": params ])
    }

    func onConnectionStateChange(_ transport: Transport!, connectionState: String!) {
        l.verbose("** Send Con change: \(connectionState ?? "n/a")")
    }
}

// callsite

self.sendTransport = self.device.createSendTransport(sendTransportHandler.delegate,
                                                             id: id,
                                                             iceParameters: iceParams,
                                                             iceCandidates: iceCandidates,
                                                             dtlsParameters: dtlsParams)

This doesn't happen if I just declare my RoomClient that's already containing the sendTransport as SendTransportListener and handle the delegate methods there.

I see the associated promise error coming up a lot in other issues here. Wrapping in async queues didn't really help.

67 might be related.

[xplatsolutions]

Definitely related. In general there is also https://github.com/ethand91/mediasoup-ios-client/issues/68 which causes issues before firing onProduce.

[ethand91]

The promise object seems to be destructed, basically it considers itself finished... This may be caused when onProduce is called on thread 1, but you are returning the producerId on Thread 2?

[MartinP7r]

Yes, they were on different queues. The onProduce is called on default-qos and the socket.io request is being returned on a designated queue since all it's requests should be handled over the same queue (see https://nuclearace.github.io/Socket.IO-Client-Swift/Classes/SocketManager.html#/c:@M@SocketIO@objc(cs)SocketManager(py)handleQueue ).

I've rewriten the handler to run sequentially (with a semaphore) in order to return on the same thread. I'll observe whether this resolves the problem.

[ethand91]

Did you solve this if you handled it on the same thread?

[MartinP7r]

Sorry for closing without update. I thought that the threading might have been it.

But, this morning I figured out, that I had messed up my socket.io requests. They were on a non-serial thread after all. Fixing that solved the problem.
My onProduce() still looks like in the source above and I'm receiving the delegate call for onProduce() on com.apple.root.default-qos and returning the callback on the serial thread.
The error did not show up since.

[xplatsolutions]

Would you mind sharing the serial queue code snippet?

[MartinP7r]

let handleQueue = DispatchQueue(label: "socket.io-serial",
                                            qos: .userInitiated,
                                            attributes: [])

let manager = SocketManager(socketURL: SERVER_ADDR, config: YOUR_CONFIG)
manager.handleQueue = handleQueue
manager.defaultSocket.connect()
// ... after this. manager.defaultSocket.emitWithAck() etc. should run on socket.io-serial

edit: no attributes selected since serial is the default nowadays.

[tanaymondal]

I am facing the same problem, using protoo backend server [ERROR] transport_wrapper::+[TransportWrapper nativeProduce:listener:track:encodings:codecOptions:appData:]() | The associated promise has been destructed prior to the associated state becoming ready.

Any solution?

[xplatsolutions]

I personally gave up, it seems that multithreading is not handled correctly and nobody has the time to focus on mediasoup iOS library. I tried various approaches to sync threads but even if you do that it is not guaranteed that your callback will be invoked on the same thread that mediasoup handler is called.

[tanaymondal]

@xplatsolutions Then how this framework is working for others? Are they using mediasoup-sample-server by ethand91?

[xplatsolutions]

I am not sure how it is working for others, I haven't seen/know any production app using this. I have seen in mediasoup forum people giving up dealing with iOS native and moving to React Native. Not saying it's not working but definitely didn't work for me. As soon as you start messing with async and multithreading you will hit this issue which is vital for the stream to work.

[tanaymondal]

@xplatsolutions The promise issue is resolved. I am able to produce audio and can hear it from mediasoup web demo but there is some issue with producing video. On mediasoup demo server, the video resolution is showing as 0x0.

[xplatsolutions]

So you can safely invoke the callback() inside the produce handler from any thread or did you come up with some kind of synchronization? About the video resolution problem, I'd check your constraints passed to mediasoup. Make sure your RTP parameters are the correct ones.

[tanaymondal]

Finally, all issues are resolved. The demo app is working fine with the media soup demo server.

[cavoixanh]

Finally, all issues are resolved. The demo app is working fine with the media soup demo server.

@tanaymondal If you can share sample ios demo? I also finished demo app but It still get bugs in third, fourth call. Which framework version do you using? or you build framework from code yourself?

[xplatsolutions]

Same here, we'd like some more info or the sample working, the promise callback handler error is a nasty one. Thanks for the help in advance.

[cavoixanh]

@xplatsolutions I saw you research in 3-4 months. Can't you finish a complete application? Our company is intending use mediasoup for production application(Web/IOS/Android). Do you think it is ok?

[xplatsolutions]

I went full web responsive at the moment. Not very confident in this library since I couldn't make a simple sample work in native iOS. Never tried native android.

[cavoixanh]

I went full web responsive at the moment. Not very confident in this library since I couldn't make a simple sample work in native iOS. Never tried native android.

Thank you for your review.

[MartinP7r]

@tanaymondal did you get it working consistently with multiple users and rooms?

[cavoixanh]

Hi @MartinP7r ,

Did you get some issues related thread? In each call, I saw memory increase 40Mb although end call (close sendtransport successfully, disconnected socket) and hang local video in second call. Also created about 20 new threads each call. Finally, leak memory and app take more than 200mb after 4-5 calls. CPU took about 70%.

If you also get issues? or root cause is at my code?

btw: if you can join skype group for discuss, please join : https://join.skype.com/UPQ6hy6bWnQ5

[MartinP7r]

@cavoixanh yes, the issues you describe are probably the same I had. closing transports didn't free up memory and added around 40mb every time until the app crashes.

I'm currently not actively working on it anymore b/c of lack of time, sorry I can't be of more help to you.

[darnfish]

Any updates here, will this ever be fixed?