twilio / voice-quickstart-ios

Twilio Voice Quickstart for iOS with Swift
MIT License
183 stars 97 forks source link

iOS Energy Crash with TwilioSDK 6 #481

Closed benford-servicem8 closed 3 years ago

benford-servicem8 commented 3 years ago

Hi,

We have an iOS app running the TwilioSDK and are receiving 10's of Energy based crashes daily which appear to be caused by the TwilioSDK using too much CPU during calls while the app is in the background.

Eg. -High CPU Usage Termination (9 seconds cpu time over 9 seconds (100% cpu average), exceeding limit of 60% cpu over 15 seconds) -High CPU Usage Termination (90 seconds cpu time over 90 seconds (100% cpu average), exceeding limit of 50% cpu over 180 seconds) -High CPU Usage Termination (90 seconds cpu time over 158 seconds (57% cpu average), exceeding limit of 50% cpu over 180 seconds)

The backtraces vary, but all appear to belong to the SipStack during a call. image

Is there anything we can do to reduce the amount of CPU being consumed during calls so the iOS watchdog doesn't terminate our process?

Thanks Ben

bobiechen-twilio commented 3 years ago

@benford-servicem8 Thanks for reporting this. We will perform some instrumentation and gather some debugging data then get back to you. Just to confirm - is it as simple as putting the app to the background during a call and does this happen only when the app is not in the foreground, i.e. does this also happen when the device is lock?

benford-servicem8 commented 3 years ago

@bobiechen-twilio Thanks, we haven't been able to reproduce the crash, it seems energy based crashes are disabled when the debugger is attached. When we profile the app while debugging the CPU doesn't exceed 5% during a call, however we have logging that confirms that the CPU on some calls is much higher.

We estimate that about 1% of calls are experiencing the issue. The most common report we are receiving from customers is that the call will drop 10 seconds after answering, which matches the "exceeding limit of 60% cpu over 15 seconds" rule. The iOS energy watchdog only applies when the app is in the background, and i would assume lock screen as well.

benford-servicem8 commented 3 years ago

@bobiechen-twilio We've managed to capture TwilioSDK debug logs just prior to the crash occurring, is it possible for me to open a support ticket for you to review the log?

bobiechen-twilio commented 3 years ago

@benford-servicem8 you can send it to me (see my profile) so we can take a look directly without going through the support ticket process.

bobiechen-twilio commented 3 years ago

Hi @benford-servicem8

Just want to let you know that we haven't forgotten about this one and are actively investigating with different profiling tools. Hope to update you soon.

bobiechen-twilio commented 3 years ago

I've CPU-profiled our internal test app but didn't see the jump of CPU usage when moving the app to the background mid call under different network connection types (Wifi/cellular), since the screenshot from the description seems to indicate the signaling stack was causing the high CPU. I did see a couple of WebRTC audio related threads working pretty hard but they managed to keep at ~50% of CPU load.

I also checked the env data from the user feedback but it doesn't seem to be caused by older models of devices since they are using a very recent model (iPhone 12). Is there any chance we can gather more information regarding device models or network environment of the crashes?

Screen Shot 2021-10-20 at 4 14 53 PM
benford-servicem8 commented 3 years ago

Hi @bobiechen-twilio ,

We've been able to to successfully reproduce the issue, which allowed us to identify our bug which was causing it to occur.

We were incorrectly retaining the TVOCall object after call completion in some cases. This caused a build up of TwilioSDK related threads to lay suspended in the app after each call, and then when an inbound call was received there is a % chance that a past call thread would lock consuming 100% cpu, causing the iOS energy watchdog to terminate the app during the call.

Fixing the retain issue on the TVOCall object allows the Twilio threads to successfully complete, and prevent the high cpu on incoming call issue.

Thanks for your help getting this resolved,

Cheers, Ben

bobiechen-twilio commented 3 years ago

Thanks for sharing the debugging experience @benford-servicem8!