voxeet / voxeet-sdk-ios

The Dolby.io Communications SDK for iOS.
https://docs.dolby.io/communications-apis/docs/ios-overview
Other
3 stars 0 forks source link

dolbyVoice conference issue when joining with disabled audio #64

Open vodemn opened 1 year ago

vodemn commented 1 year ago

When joining a dolbyVoice conference with disabled audio, ConferenceService fires 2 events: participantAdded and streamAdded. In the first event participant has an empty streams array (which corresponds to joinOptions.constraints) and the second one adds an audio stream, even though join constraint for audio was set to false.

...
let constraints = VTJoinOptionsConstraints()
constraints.audio = false
constraints.video = false // this value doesn't affect the described behavior
joinOptions.constraints = constraints
VoxeetSDK.shared.conference.join(conference: conference, options: joinOptions) {} 

This only happens with dolbyVoice conferences.

Expected Behaviour

Audio stream should not be added when joining dolbyVoice conference with disabled audio.

Specifications

vodemn commented 1 year ago

So when joining a DolbyVoice conference with audio disabled I noticed this error in log:

(dvc_conference_impl.cc:339): SetMute failed in direction == webrtc_integration::DvcDirection::OUTPUT || mute == true || listener_mode_ == false with status false
vodemn commented 1 year ago
Joining with audio ``` (audio_device_generic.cc:18): BuiltInAECIsAvailable: Not supported on this platform (audio_device_generic.cc:28): BuiltInAGCIsAvailable: Not supported on this platform (audio_device_generic.cc:38): BuiltInNSIsAvailable: Not supported on this platform (connection_observer_on_track.hpp:30): --- SignalingChange (audio_device_generic.cc:18): BuiltInAECIsAvailable: Not supported on this platform (audio_device_generic.cc:28): BuiltInAGCIsAvailable: Not supported on this platform (audio_device_generic.cc:38): BuiltInNSIsAvailable: Not supported on this platform (utils.hpp:15): 0 (media_engine.hpp:195): --- Native add stream (media_engine.hpp:195): --- Native add stream (connection_observer_on_track.hpp:30): --- SignalingChange 2022-10-19 13:38:11.307959+0300 Runner[19927:5439342] ###### ###### ###### WebRTC callStackSymbols ###### ###### ###### (connection_observer_on_track.hpp:75): --- OnIceConnectionChanged (audio_device_generic.cc:18): BuiltInAECIsAvailable: Not supported on this platform (audio_device_generic.cc:28): BuiltInAGCIsAvailable: Not supported on this platform (audio_device_generic.cc:38): BuiltInNSIsAvailable: Not supported on this platform ```
Joining without audio ``` (audio_device_generic.cc:18): BuiltInAECIsAvailable: Not supported on this platform (audio_device_generic.cc:28): BuiltInAGCIsAvailable: Not supported on this platform (audio_device_generic.cc:38): BuiltInNSIsAvailable: Not supported on this platform (connection_observer_on_track.hpp:30): --- SignalingChange (audio_device_generic.cc:18): BuiltInAECIsAvailable: Not supported on this platform (audio_device_generic.cc:28): BuiltInAGCIsAvailable: Not supported on this platform (audio_device_generic.cc:38): BuiltInNSIsAvailable: Not supported on this platform (utils.hpp:15): 0 (media_engine.hpp:195): --- Native add stream (media_engine.hpp:195): --- Native add stream (connection_observer_on_track.hpp:30): --- SignalingChange 2022-10-19 13:39:26.569997+0300 Runner[19927:5439342] ###### ###### ###### WebRTC callStackSymbols ###### ###### ###### (connection_observer_on_track.hpp:75): --- OnIceConnectionChanged (dvc_conference_impl.cc:339): SetMute failed in direction == webrtc_integration::DvcDirection::OUTPUT || mute == true || listener_mode_ == false with status false (audio_device_generic.cc:18): BuiltInAECIsAvailable: Not supported on this platform (audio_device_generic.cc:28): BuiltInAGCIsAvailable: Not supported on this platform (audio_device_generic.cc:38): BuiltInNSIsAvailable: Not supported on this platform ```
IgorPaliashchuk commented 1 year ago

Hello, any updates on that?

mohsindlb commented 1 year ago

@IharPaliashchuk Is this issue happened with voxeet-uxkit-ios or on Voxeet-sdk-ios? as you have mentioned voxeet-uxkit-ios it has latest version 1.7.4

If you are referring to voxeet-sdk-ios v 3.4.0 then we have stopped supporting this version [2023-03-31 end of support]. Please refer our SDK support page for more info.

I would recommend you to upgrade to the latest supported version and try again.

IgorPaliashchuk commented 1 year ago

@mohsindlb we use voxeet-sdk-ios v3.8.0

I see similar logs to what was posted before. Crashes sometimes occur when a user joins a conference with audio enabled.

First
``` (audio_device_generic.cc:18): BuiltInAECIsAvailable: Not supported on this platform (audio_device_generic.cc:28): BuiltInAGCIsAvailable: Not supported on this platform (audio_device_generic.cc:18): BuiltInAECIsAvailable: Not supported on this platform (audio_device_generic.cc:28): BuiltInAGCIsAvailable: Not supported on this platform (connection_observer_on_track.hpp:30): --- SignalingChange (audio_device_generic.cc:18): BuiltInAECIsAvailable: Not supported on this platform (audio_device_generic.cc:28): BuiltInAGCIsAvailable: Not supported on this platform (audio_device_generic.cc:18): BuiltInAECIsAvailable: Not supported on this platform (audio_device_generic.cc:28): BuiltInAGCIsAvailable: Not supported on this platform (utils.hpp:15): 0 (media_engine.hpp:191): --- Native add stream (media_engine.hpp:191): --- Native add stream (connection_observer_on_track.hpp:30): --- SignalingChange 2023-07-04 18:11:05.454529+0400 Runner[3078:992791] [avae] AVAEInternal.h:76 required condition is false: [AVAudioIONodeImpl.mm:1229:SetOutputFormat: (format.sampleRate == hwFormat.sampleRate)] 2023-07-04 18:11:05.455089+0400 Runner[3078:992791] *** Terminating app due to uncaught exception 'com.apple.coreaudio.avfaudio', reason: 'required condition is false: format.sampleRate == hwFormat.sampleRate' *** First throw call stack: (0x1d341e248 0x1cc7dfa68 0x1d358a3b8 0x1e9ad6624 0x1e9ba7e4c 0x1e9ad588c 0x1e9acece0 0x1e9bc6070 0x1064fc868 0x1064fc544 0x10d776ae0 0x10d22558c 0x109990598 0x10999204c 0x1099a2800 0x1099a2344 0x1d34aea08 0x1d3490368 0x1d34951e4 0x20c2b5368 0x1d5944d88 0x1d59449ec 0x104b83544 0x1f17b9948) libc++abi: terminating with uncaught exception of type NSException *** Terminating app due to uncaught exception 'com.apple.coreaudio.avfaudio', reason: 'required condition is false: format.sampleRate == hwFormat.sampleRate' terminating with uncaught exception of type NSException ```
Second
``` SignalingChange (audio_device_generic.cc:18): BuiltInAECIsAvailable: Not supported on this platform (audio_device_generic.cc:28): BuiltInAGCIsAvailable: Not supported on this platform (audio_device_generic.cc:18): BuiltInAECIsAvailable: Not supported on this platform (audio_device_generic.cc:28): BuiltInAGCIsAvailable: Not supported on this platform (utils.hpp:15): 0 (media_engine.hpp:191): --- Native add stream (media_engine.hpp:191): --- Native add stream (connection_observer_on_track.hpp:30): --- SignalingChange 2023-07-04 18:05:05.895116+0400 Runner[3063:985686] [avae] AVAEInternal.h:109 [AVAudioEngineGraph.mm:1397:Initialize: (err = AUGraphParser::InitializeActiveNodesInInputChain(ThisGraph, *GetInputNode())): error -10868 audio_streamer/SwiftAudioStreamerPlugin.swift:82: Fatal error: 'try!' expression unexpectedly raised an error: Error Domain=com.apple.coreaudio.avfaudio Code=-10868 "(null)" UserInfo={failed call=err = AUGraphParser::InitializeActiveNodesInInputChain(ThisGraph, *GetInputNode())} 2023-07-04 18:05:05.896725+0400 Runner[3063:985686] audio_streamer/SwiftAudioStreamerPlugin.swift:82: Fatal error: 'try!' expression unexpectedly raised an error: Error Domain=com.apple.coreaudio.avfaudio Code=-10868 "(null)" UserInfo={failed call=err = AUGraphParser::InitializeActiveNodesInInputChain(ThisGraph, *GetInputNode())} ```
mohsindlb commented 1 year ago

@IharPaliashchuk Thanks for confirming. I will check and get back to you.

mohsindlb commented 1 year ago

@IharPaliashchuk I tried to reproduce the scenario with sdk v 3.8.0 but my application does not crashes. Do you have any specific steps to reproduce this? I tried with our getting started app. Can you please share your code snippet to look into it?

However I can see the audio stream object in participant and stream event, even though we join the conference without audio. I will check on this and get back to you.

IgorPaliashchuk commented 1 year ago

@mohsindlb Thank you the reply. Actually, our app is a Flutter app and we only use native code for VoxeetSDK communication. We don't use any special settings and follow the getting started app.

Let me send some code which is related to VoxeetSDK.

let keys = call.arguments as! Array<String>
VoxeetSDK.shared.initialize(consumerKey: keys[0], consumerSecret: keys[1])
VoxeetSDK.shared.session.delegate = self.sessionEventsHandler
VoxeetSDK.shared.conference.delegate = self.conferenceEventsHandler
VoxeetSDK.shared.notification.delegate = self.notificationEventsHandler

or

let token = call.arguments as! String
VoxeetSDK.shared.initialize(accessToken: token, refreshTokenClosureWithParam: { [weak self] closure, _ in
    self?.sdkMethodChannel!.invokeMethod("getRefreshToken", arguments: nil, result: { result -> Void in
        let token = result is String ? result as? String : nil
        closure(token)
    })
})
VoxeetSDK.shared.session.delegate = self.sessionEventsHandler
VoxeetSDK.shared.conference.delegate = self.conferenceEventsHandler
VoxeetSDK.shared.notification.delegate = self.notificationEventsHandler

then

VoxeetSDK.shared.session.open(info: info) { self._handleError($0, result) }
let args = call.arguments as! Dictionary<String, Any?>
let createArgs = args["create"] as! Dictionary<String, Any>
let createOptions = VTConferenceOptions()
createOptions.alias = createArgs["alias"] as? String
createOptions.params.audioOnly = createArgs["audioOnly"] as! Bool
createOptions.params.dolbyVoice = createArgs["dolbyVoice"] as! Bool

VoxeetSDK.shared.conference.create(options: createOptions) { conference in
    let joinArgs = args["join"] as! Dictionary<String, Any?>
    self._joinConference(
       conference,
       result,
       hasAudio: joinArgs["audio"] as! Bool,
       hasVideo: joinArgs["video"] as! Bool
    )
} fail: {
    self._handleError($0, result)
}

private func _joinConference(
    _ conference: VTConference,
    _ result: @escaping FlutterResult,
    hasAudio: Bool,
    hasVideo: Bool
) {
    let joinOptions = VTJoinOptions()
    let constraints = VTJoinOptionsConstraints()
    constraints.audio = conference.params.dolbyVoice ? true : hasAudio
    constraints.video = hasVideo
    joinOptions.constraints = constraints

    VoxeetSDK.shared.conference.join(conference: conference, options: joinOptions) {
       if (conference.params.dolbyVoice && !hasAudio) { //fix to do not have crashes
           VoxeetSDK.shared.audio.local.stop {_ in }
       }
       result($0.toJson())
    } fail: {
       self._handleError($0, result)
    }
}
case "startAudio":
   VoxeetSDK.shared.audio.local.start { self._handleError($0 as NSError?, result) }
case "stopAudio":
    VoxeetSDK.shared.audio.local.stop { self._handleError($0 as NSError?, result) }
case "startVideo":
    VoxeetSDK.shared.video.local.start { self._handleError($0 as NSError?, result) }
case "stopVideo":
    VoxeetSDK.shared.video.local.stop { self._handleError($0 as NSError?, result) }

I would like to mention that everything works on each platform (joining a conference and inviting participants, enabling/disabling audio/video and leaving), only ios has crushes sometimes if user joins with enabled audio.

FabienLavocat commented 1 year ago

@IharPaliashchuk I have created #70 on your behalf as the issue you are describing is when a user tries to join a conference with Audio enabled. This issue is about having a stream object when audio is disabled.