Not able to handle calls in webrtc/js

[HugoHeneault]

I've setup cordova-plugin-callkit on my 2019 project which was broken due to iOS13 CallKit changes.

I'm receiving VOIP notifications. Callkit screen appears and my js code reacts to the answer event.

I then have 2 issues :

1. With opened app, no sound goes in/out the iPhone
2. With phone locked, js code seems to be paused after a few secs and I have to unlock it to be able to run my call-logic system (still with no sound).

Has anyone managed to run a js-based call system? Or the whole call-part has to be natively run?

Thanks a lot for your plugin @mattkhaw !

[mattkhaw]

I'm using this plugin for my app also. I only have this to say.

1. I've experienced this problem before but only on newer iOS versions. iOS 12 is fine though. The problem lies in the webrtc library itself. It has something to do with the audio session setup.

2. This is a normal behaviour even for other popular apps. The only solution is to unlock the phone yourself once you've answered the call. That's why there's an additional button for you to open your app and the logo is interchangeable.

This is the only solutions that I've encountered. The first issue is the worst since it is more generic and costs me a lot of sleepless nights. The second one is basically Apple's software restriction which you can't do anything about it.

Hope this helps and good luck!

[HugoHeneault]

• I've experienced this problem before but only on newer iOS versions. iOS 12 is fine though. The problem lies in the webrtc library itself. It has something to do with the audio session setup.

So you're having this problem on iOS 14.x too? 😅

[HugoHeneault]

2. This is a normal behaviour even for other popular apps. The only solution is to unlock the phone yourself once you've answered the call. That's why there's an additional button for you to open your app and the logo is interchangeable.

Also are you implying that I should change the logo to advice the user to unlock its phone? :D

Thanks for your replies!

[mattkhaw]

• I've experienced this problem before but only on newer iOS versions. iOS 12 is fine though. The problem lies in the webrtc library itself. It has something to do with the audio session setup.

So you're having this problem on iOS 14.x too? 😅

2. This is a normal behaviour even for other popular apps. The only solution is to unlock the phone yourself once you've answered the call. That's why there's an additional button for you to open your app and the logo is interchangeable.

Also are you implying that I should change the logo to advice the user to unlock its phone? :D

Thanks for your replies!

1. Yes, I've already stated where the problem is. Just check that part.

2. You can refer to this article -> https://9to5mac.com/2018/10/29/snapchat-calling-callkit-integration/

It's totally up to you what you want to do with your current app but AFAIK, there's no way to bypass this.

[mattkhaw]

Issue closed due to inactivity.

[HugoHeneault]

After a few nights working on making this plugin work for locked-cases, I finally got some Swift courses and created my own native callkit/websocket plugin which works perfectly. 😅

I fear that if you guys need to get locked iPhone w/ callkit & webrtc the only solution is to do the same. I can help if needed :-)

[rgustavsson]

@HugoHeneault I would be very happy to get some help to fix my issue with "With opened app, no sound goes in/out the iPhone". I had this working at some point in 2021, but now when I answer the call via Callkit and my app opens, I connect to the WebRTC call via JS, then there is no outgoing sound.

[mattkhaw]

@HugoHeneault I would be very happy to get some help to fix my issue with "With opened app, no sound goes in/out the iPhone". I had this working at some point in 2021, but now when I answer the call via Callkit and my app opens, I connect to the WebRTC call via JS, then there is no outgoing sound.

As I've mentioned before, for my case it is the WebRTC plugin which is causing the issue. I've already informed them about it and they have solved the issue with the fix I've recommended. If you are using the latest version, then it might be some other place which is causing the issue.

After a few nights working on making this plugin work for locked-cases, I finally got some Swift courses and created my own native callkit/websocket plugin which works perfectly. 😅

I fear that if you guys need to get locked iPhone w/ callkit & webrtc the only solution is to do the same. I can help if needed :-)

I think for locked cases, I don't think there's an exposed API for us to use to unlock programmatically. Maybe you can share what have you done for that case. Might help others if they need it (me included XD).

[rgustavsson]

@mattkhaw Many thanks for your response. Yes I followed the history and didn't really want to bug you about this anymore, hence the tagging on hugo :) But I appreciate you answering.

Now that Apple/iOS has full support for WebRTC in the WKWebview we don't use any extra cordova plugins or pods or any specific WebRTC library, we simply make use of the Javascript WebRTC API's

We used to use cordova-plugin-iosrtc for WebRTC integration but have now dropped it, and with this I think we might have dropped some logic that handled audio configuration, such as shared audio sessions.

After a lot of investigation I think I believe that the Callkit plugin/logic takes control of the audio session and when the app tries to establish the WebRTC session we get the following error:

[WebRTC] Failed to activate audio session with error: 561017449.

My assumption seems to be the same as your initial lead, the audio session setup.

I'm a fullstack dev but not so great in objective-C and Swift so I was hoping that hugo would have some code to share for his fix :) But for now I will dig into the cordova-plugin-iosrtc and see how they handled the audio session setup, maybe I can fork a callkit repo and apply whatever they did there.

Again - thanks for answering!

[HugoHeneault]

@HugoHeneault I would be very happy to get some help to fix my issue with "With opened app, no sound goes in/out the iPhone". I had this working at some point in 2021, but now when I answer the call via Callkit and my app opens, I connect to the WebRTC call via JS, then there is no outgoing sound.

Hello @rgustavsson ! From what I found a few months ago, it wasn't possible to share the webrtc audio session started by callkit with js. What I did was a native iOS plugin which handle the webrtc part.

Do you connect yo wertc call via js and make the webrtc call natively or in js ?

[rgustavsson]

Thanks for the feedback @HugoHeneault

I actually use Openvidu for the WebRTC parts, and that's fully JS. Works very well without involving callkit :)

Ok ok. I don't want to do the WebRTC call natively :) But yeah. I might not have an option..

Or I go back to using cordova-plugin-iosrtc which had much of it all natively..

Many thanks for the answers guys!

[HugoHeneault]

I hope you'll get things sorted out! It was quite painful on my side so I finally decided to move to native! But there might be an option.

Happy coding!

[mattkhaw]

@mattkhaw Many thanks for your response. Yes I followed the history and didn't really want to bug you about this anymore, hence the tagging on hugo :) But I appreciate you answering.

Now that Apple/iOS has full support for WebRTC in the WKWebview we don't use any extra cordova plugins or pods or any specific WebRTC library, we simply make use of the Javascript WebRTC API's

We used to use cordova-plugin-iosrtc for WebRTC integration but have now dropped it, and with this I think we might have dropped some logic that handled audio configuration, such as shared audio sessions.

After a lot of investigation I think I believe that the Callkit plugin/logic takes control of the audio session and when the app tries to establish the WebRTC session we get the following error:

[WebRTC] Failed to activate audio session with error: 561017449.

My assumption seems to be the same as your initial lead, the audio session setup.

I'm a fullstack dev but not so great in objective-C and Swift so I was hoping that hugo would have some code to share for his fix :) But for now I will dig into the cordova-plugin-iosrtc and see how they handled the audio session setup, maybe I can fork a callkit repo and apply whatever they did there.

Again - thanks for answering!

I see. I didn't know WkWebView natively supports WebRTC. At that time, there isn't so we have to use the plugin to achieve this.

I do believe that the issue is this function right here:

- (void)setupAudioSession { @try { AVAudioSession *sessionInstance = [AVAudioSession sharedInstance]; [sessionInstance setCategory:AVAudioSessionCategoryPlayAndRecord error:nil]; [sessionInstance setMode:AVAudioSessionModeVoiceChat error:nil]; NSTimeInterval bufferDuration = .005; [sessionInstance setPreferredIOBufferDuration:bufferDuration error:nil]; [sessionInstance setPreferredSampleRate:44100 error:nil]; NSLog(@"Configuring Audio"); } @catch (NSException *exception) { NSLog(@"Unknown error returned from setupAudioSession"); } return; }

If you are omitting the cordova-plugin-iosrtc plugin, then I think it is safe to say that you can try disabling setupAudioSession function and see whether it changes or not but from my experience, it is kinda required to configure the AVAudioSession to some extent. Maybe can redirect this to when your app has connected successfully via WebRTC instead of configuring it when the call is initiated in CallKit?

From what I know, the part that handles the audio for cordova-plugin-iosrtc is PluginRTCAudioController.swift but my situation is because of the AVAudioSession mode, which I've explained in this link -> https://github.com/cordova-rtc/cordova-plugin-iosrtc/issues/632#issuecomment-771453038.

What @HugoHeneault said about sharing audio session is true. You can only have 1 audio session so, I assumed that whenever you setup the audio session, whichever comes last will override the previous one.

Now, I'm still curious about the locked phone situation. Maybe can share what's the issue that you are facing and how you solved it?