Audio Session related issue in CallKit

[StarTeleLogic]

I am facing issue in CallKit integration. Actually issue is coming after updating OS version to iOS 11.01. I have implemented CallKit as per Apple guided in their SpeakerBox example step by step.

Very first time It works fine but in subsequent runs It fails to give voice in earphone. The exact issue is that there is an inconsistency related to audio session means I am not able to use mic to catch voice and the user on other end not able to listen my voice.

All the delegates provided in Provider Class are calling very well. Audio Class instance is also available each time I run. Only issue seems with the creation of Audio session using It's shared instance. I notices that It fires AVAudioSessionRouteChangeNotification frequently and switch-case goes in the case of AVAudioSessionRouteChangeReasonCategoryChange. When I print category type- It gives right selected category named AVAudioSessionCategoryPlayAndRecord.

Let you know that If I put break point in switch-case then It works fine. May be at that point It got enough time to create Audio Session to catch the voice through mic.

Sharing the code where I am setting up audio session:

• (void)setupAudioSession { try { // Configure the audio session AVAudioSession sessionInstance = [AVAudioSession sharedInstance]; // we are going to play and record so we pick that category NSError error = nil; [sessionInstance setCategory:AVAudioSessionCategoryPlayAndRecord error:&error]; XThrowIfError((OSStatus)error.code, "couldn't set session's audio category");

  // set the mode to voice chat
  [sessionInstance setMode:AVAudioSessionModeVoiceChat error:&error];
  XThrowIfError((OSStatus)error.code, "couldn't set session's audio mode");
  
  // set the buffer duration to 5 ms
  NSTimeInterval bufferDuration = .005;
  [sessionInstance setPreferredIOBufferDuration:bufferDuration error:&error];
  XThrowIfError((OSStatus)error.code, "couldn't set session's I/O buffer duration");
  
  // set the session's sample rate
  [sessionInstance setPreferredSampleRate:44100 error:&error];
  XThrowIfError((OSStatus)error.code, "couldn't set session's preferred sample rate");
  
  // add interruption handler
  [[NSNotificationCenter defaultCenter] addObserver:self
                                           selector:@selector(handleInterruption:)
                                               name:AVAudioSessionInterruptionNotification
                                             object:sessionInstance];
  
  // we don't do anything special in the route change notification
  [[NSNotificationCenter defaultCenter] addObserver:self
                                           selector:@selector(handleRouteChange:)
                                               name:AVAudioSessionRouteChangeNotification
                                             object:sessionInstance];
  
  // if media services are reset, we need to rebuild our audio chain
  [[NSNotificationCenter defaultCenter]   addObserver:    self
                                           selector:      @selector(handleMediaServerReset:)
                                               name:      AVAudioSessionMediaServicesWereResetNotification
                                             object:      sessionInstance];

  }

  catch (CAXException &e) { NSLog(@"Error returned from setupAudioSession: %d: %s", (int)e.mError, e.mOperation); } catch (...) { NSLog(@"Unknown error returned from setupAudioSession"); }

  return; }

Below are the method with switch-cases which gets called in case of notification name :AVAudioSessionRouteChangeNotification is fired:

• (void)handleRouteChange:(NSNotification )notification { UInt8 reasonValue = [[notification.userInfo valueForKey:AVAudioSessionRouteChangeReasonKey] intValue]; AVAudioSessionRouteDescription routeDescription = [notification.userInfo valueForKey:AVAudioSessionRouteChangePreviousRouteKey];

  NSLog(@"Route change:"); switch (reasonValue) { case AVAudioSessionRouteChangeReasonNewDeviceAvailable: NSLog(@" NewDeviceAvailable"); break; case AVAudioSessionRouteChangeReasonOldDeviceUnavailable: NSLog(@" OldDeviceUnavailable"); break; case AVAudioSessionRouteChangeReasonCategoryChange: NSLog(@" CategoryChange"); NSLog(@" New Category: %@", [[AVAudioSession sharedInstance] category]); [self setupAudioSession]; break; case AVAudioSessionRouteChangeReasonOverride: NSLog(@" Override"); break; case AVAudioSessionRouteChangeReasonWakeFromSleep: NSLog(@" WakeFromSleep"); break; case AVAudioSessionRouteChangeReasonNoSuitableRouteForCategory: NSLog(@" NoSuitableRouteForCategory"); break; default: NSLog(@" ReasonUnknown"); }

  NSLog(@"Previous route:\n"); NSLog(@"%@", routeDescription); }

PLEASE NOTE THAT EVERY THING WORKS FINE IF I PUT BREAK POINT IN ABOVE METHOD'S CASE- AVAudioSessionRouteChangeReasonCategoryChange !!!

[alqa-dash]

Hi! Recently I faced with the same issue. I use liblinphone sdk in my own project (I use CallKit too) and discovered that there is problem with sound when incoming call is on the ios 11 part: for short period of time after pick up the phone I hear sound and then silence. When a call is initiated on ios 11 - everything is ok with audio. Unfortunately I have no time to check it deeper. With ios 10 audio is stable.

[alqa-dash]

Hey, I've just found this:

(https://developer.apple.com/library/content/releasenotes/General/WhatsNewIniOS/Articles/iOS_11_0.html) New in iOS 11.0 - AirPlay 2. Improved AirPlay reliability for some audio playback interfaces in AV Foundation. To take advantage of the increased reliability, play audio using AVPlayer or the new AVSampleBufferAudioRenderer object. Added multiple speaker support to AirPlay for long-form audio, such as music and podcasts. To mark your application as presenting long-form audio, invoke the AVAudioSession method setCategory(_:mode:routeSharingPolicy:options:) and use AVAudioSessionRouteSharingPolicyLongForm as the parameter value. Added AVRoutePickerView to the AVKit framework and AVRouteDetector to the AVFoundation framework for enabling users to choose the route for playing content when multiple routes are available. Use AVRouteDetector to determine if multiple routes are available when route detection is enabled. If multiple routes are available, use AVRoutePickerView to present an interface for the user to choose the routes.

Did not check yet.

[alqa-dash]

Looks like this problem is connected to AirPlay 2 functionality in iOS 11. The only thing I found is when I use setCategory(_:mode:routeSharingPolicy:options:) sometimes I can hear sound in first call after app restart but definitely not in second and following calls. In app log I saw following:

2017-10-15 22:54:04.482293 [aurioc] 892: failed: '!pri' (enable 3, outf< 1 ch, 8000 Hz, Int16> inf< 1 ch, 8000 Hz, Int16>) 2017-10-15 22:54:04:482 ortp-error-Cannot start audioUnit because [!pri] 2017-10-15 22:54:04:482 ortp-error-AudioUnit error for AudioUnitInitialize(card->io_unit): ret=unknown error (561017449)

Unfortunately I don't know how to handle this error. May be someone can help here?

[ghost]

Hello,

We've encountered a similar issue where the call is accepted before CallKit activates the audio session. it has been resolved by waiting didActivateAudioSession callback before accepting a call. You can look our implementation in the ProviderDelegate.m file and take a look on our wiki : https://wiki.linphone.org/xwiki/wiki/public/view/Lib/Getting%20started/iOS/#HCallKitIntegration You can test, the issue doesn't happen on the Linphone iOS app.

Regards

[alqa-dash]

Hello,

In iOS 10 everything is working fine but in iOS 11 the same code produces only video. I did all the same things: set category, mode, options, wait until didActivate finish it's job. I guess that in iOS 11 new feature "AirPlay 2" does something wrong. Unfortunately I still didn't find what exactly.

[alqa-dash]

Hey,

Seems to be the key problem that often didActivate fires too late. I increase timeout before answering the call and now audio is ok!

[ghost]

Okay, i'll close issue then.

Regards

[StarTeleLogic]

No problem yet to fix, please reopen this issue.

[ghost]

@StarTeleLogic Do you wait for didActivateAudioSession before starting the call ?

[lucifer662607004]

@ReisBenjamin I'm also having the same issue only when the users click End&Accept button in the second incoming call. Not sure if it is the same case.

I use linphone_call_accept_with_params() in didActivateAudioSession just like Linphone did. The first incoming call has no problem. When the second incoming call interrupts the first one and the users click "End&Accept" button, the first one is terminated and the second one is accepted as expect, but the audio disappears. The linphonecore stop sending any RTP packets and keeps saying something like:

: ort/rtp_putq: Queue is full. Discarding message with ts=26240 (The second call has already been established with ACK from server). I have to pause the call with linphone_core_pause_call and resume it again with linphone_core_resume_call to restore its audio. After that it will function normally. Could you give me a hint where should I start to debug this issue? Appreciate any help, thank you. Callkit interaction orders when dual incoming call: 1. reportNewIncomingCallWithUUID: 2. ProviderDelegate:performAnswerCallAction(user clicks answer button) 3. didActivateAudioSession(programmatically accept call here) 4. reportNewIncomingCallWithUUID: (second call is coming) 5. ProviderDelegate:performCallEndCallAction(user clicks End&Answer, callkit inform me to end first call) 6. requestTransaction:CXEndCallAction(Inform callkit the first call has ended. It is not necessary seems callkit already aware of that) 7. ProviderDelegate:performAnswerCallAction 8. didDeactivateAudioSession (deactivate audio after perform answer call WTF) 9. didActivateAudioSession (accept the second call here, also try to [audioSession setCategory:AVAudioSessionCategoryPlayAndRecord error:nil] again) No audio since then, in AUDIO SESSION'S RTP STATISTICS, ort/sent is ZERO.

[lucifer662607004]

@ReisBenjamin I've checked linphone-iphone's code and I accidentally found Linphone dose not report the second call to callkit. The problem is, when you hang up the first call, the audio session has been deactivated, and the second call is not able to send audio.

reproduce issue:

1. make first call to line 100.
2. make second call to 200.
3. hang up 100
4. 200 has no audio.

[ghost]

Hello,

On linphone-iphone we chose to only manage one call with callkit, if there's more than one call the other are managed by Linphone not Callkit because Callkit only allows to simultaneous calls and Linphone allows as many as possible. That's why we do not have this issue on Linphone iphone. Can you post full logs of the problem ?

Thanks

[lucifer662607004]

@ReisBenjamin Thank you. I pulled the latest version of linphone and I found linphone does not release callkit until all calls are terminated. That's a convenient and brilliant design. Maybe I should just copy that. 😂

[iJasonWade]

@alqa-dash, what did you mean you increased the timeout to get the audio to work?

[alqa-dash]

@iJasonWade, it was my attempt to fix sound issue. But it is not the right way and I dropped this option. I use didActivate instead.

[techwizardg]

i am also facing the same issue and rtp is not heard some time for audio call.Any pointer to the same?

[crperry]

Has anybody found a fix for this issue? My team is experiencing the same issue with all calls past the first. We have ensured that we are waiting for for didActivateAudioSession to be called before accepting calls.

[ravi6880]

Any body got the solution. I am using the latest source code of linphone which support to callkit , but still i am not getting the callkit notification.