search

twilio / video-quickstart-ios

Twilio Video Quickstart for iOS
https://www.twilio.com/docs/api/video
MIT License
457 stars 177 forks source link

ReplayKit problem when Callkit package is used into application #600

Closed dmichelutti closed 2 years ago

dmichelutti commented 3 years ago

Description

Broadcast Extension activated in application with Callkit framework active is not working. Only video frames are delivered, microphone is not responding.

Steps to Reproduce

  1. Start with example code https://github.com/twilio/video-quickstart-ios/tree/master/ReplayKitExample. I'm doing a simple call before streaming screenshare.
  2. I activate broadcast extension: now broadcast is working perfectly, audio (microphone) and images.
  3. Added Callkit framework in appDelegate and added a simple CallManager (e.g. https://www.raywenderlich.com/1276414-callkit-tutorial-for-ios or similar)
  4. Now first call is working perfectly, when i activate broadcast extension, microphone seems to be inactive. Also starting and stopping mic tapping on Mic button does not make anything

Expected Behavior

I will send also audio through broadcast extension.

Reproduces How Often

Disactivating Callkit framework (comment code): 100% works perfectly Activating Callkit framework (comment code): 100% fails

Logs

There are no errors in console, also activating twilio logs (TwilioVideoSDK.setLogLevel(.all)). To test this behaviour I have maked breakpoints on SampleHandler.swift class, in override func processSampleBuffer(_ sampleBuffer: CMSampleBuffer, with sampleBufferType: RPSampleBufferType) method. When Callkit framework is disabled and when screensharing is started, i have logs in all cases (.video, .audioApp, .audioMic). In case of Callkit active, I have logs only in .video and .audioApp, NOT .audioMic.

Versions

XCode 12.4, Swift 5, TwilioVideo 4.3

Video iOS SDK

4.3 via cocoapods

Xcode

XCode 12.4

iOS Version

14.4.2

iOS Device

iPad Pro (10.5-inch)

ceaglest commented 2 years ago

Hi @dmichelutti,

Sorry for the late response to your question. In a ReplayKit broadcast extension the Twilio Video SDK doesn't have control over an AVAudioSession. If the sample handler is not getting .audioMic buffers when CallKit is active it is probably intentional on Apple's part.

It might be worthy of a feedback with Apple to clarify the intended behaviour. Also, double check with the latest iOS 15 betas in case this changes the equation when it comes to ReplayKit audio.

Best, Chris