search

twilio / twilio-video-ios

Programmable Video SDK by Twilio
http://twilio.com/video
Other
64 stars 22 forks source link

[AVCaptureSession startRunning] should be called from background thread. Its currently being called from a foreground thread by Twilio SDK, now causes crash during video call on App #246

Closed Dinalli closed 1 year ago

Dinalli commented 1 year ago

Description

When using TwilioVideoSDK and backgrounding the App, when the app enters the foreground. Xcode gives the following warning

Thread Performance Checker: -[AVCaptureSession startRunning] should be called from background thread. Calling it on the main thread can lead to UI unresponsiveness

Steps to Reproduce

Xcode Version 14.1 (14B47b) TwilioVideoSDK - 5.2.1

Consume a TwilioVideo stream. When he app is backgrounded and Brough to foreground this error is thrown.

Thread Performance Checker: -[AVCaptureSession startRunning] should be called from background thread. Calling it on the main thread can lead to UI unresponsiveness PID: 886, TID: 112393 Backtrace

3 TwilioVideo 0x00000001032e90c8 -[TVICameraCapturePipeline startSession] + 52 4 TwilioVideo 0x00000001032ec09c __58-[TVICameraCapturePipeline applicationWillEnterForeground]_block_invoke + 124 5 libdispatch.dylib 0x0000000101c8605c _dispatch_client_callout + 20 6 libdispatch.dylib 0x0000000101c97884 _dispatch_lane_barrier_sync_invoke_and_complete + 172 7 TwilioVideo 0x00000001032ec010 -[TVICameraCapturePipeline applicationWillEnterForeground] + 116 8 CoreFoundation 0x00000001d64c73f0 5CDC5D9A-E506-3740-B64E-BB30867B4F1B + 226288 9 CoreFoundation 0x00000001d656e474 5CDC5D9A-E506-3740-B64E-BB30867B4F1B + 910452 10 CoreFoundation 0x00000001d6551724 5CDC5D9A-E506-3740-B64E-BB30867B4F1B + 792356 11 CoreFoundation 0x00000001d64dba08 _CFXNotificationPost + 704

Code

// Code that helps reproduce the issue

Expected Behavior

No message to be shown and the

Actual Behavior

Error message is shown

Reproduces How Often

100%

Logs

Debug level logs are helpful when investigating issues. To enable debug level logging, add the following code to your application:

2022-11-22 09:46:44.249487+0000 SharedUITest[950:128881] DEBUG:Twilio:Core: <0x282a56b20> TcmpWssTransport::onMessage (20 bytes) {"type":"heartbeat"} 2022-11-22 09:46:45.437143+0000 SharedUITest[950:128236] DEBUG:Twilio:[Platform](Thread 0x0x101a4f4e0): Application will enter foreground. 2022-11-22 09:46:45.437258+0000 SharedUITest[950:128236] INFO:Twilio:[Platform](Thread 0x0x101a4f4e0): Manually restarting interrupted session. 2022-11-22 09:46:45.438520+0000 SharedUITest[950:128236] DEBUG:Twilio:[Platform](Thread 0x0x101a4f4e0): Rendering inactive scene at 30 fps. 2022-11-22 09:46:45.438620+0000 SharedUITest[950:128236] DEBUG:Twilio:[Platform](Thread 0x0x101a4f4e0): Rendering inactive scene at 30 fps. 2022-11-22 09:46:45.439770+0000 SharedUITest[950:128587] INFO:Twilio:Platform: AVCaptureSession interrupted with reason: 1 2022-11-22 09:46:45.444199+0000 SharedUITest[950:128587] INFO:Twilio:Platform: ... Capture pipeline did stop. 2022-11-22 09:46:45.444763+0000 SharedUITest[950:128587] DEBUG:Twilio:Platform: Teardown video output pipeline ... 2022-11-22 09:46:45.446522+0000 SharedUITest[950:128587] DEBUG:Twilio:Platform: Finished video output pipeline teardown. 2022-11-22 09:46:45.446581+0000 SharedUITest[950:128587] DEBUG:Twilio:Platform: Video pipeline did finish running 2022-11-22 09:46:45.448405+0000 SharedUITest[950:128587] INFO:Twilio:Platform: AVCaptureSession stopped running Thread Performance Checker: -[AVCaptureSession startRunning] should be called from background thread. Calling it on the main thread can lead to UI unresponsiveness PID: 950, TID: 128236 Backtrace

3 TwilioVideo 0x0000000102a090c8 -[TVICameraCapturePipeline startSession] + 52 4 TwilioVideo 0x0000000102a0c09c __58-[TVICameraCapturePipeline applicationWillEnterForeground]_block_invoke + 124 5 libdispatch.dylib 0x00000001013f605c _dispatch_client_callout + 20 6 libdispatch.dylib 0x0000000101407884 _dispatch_lane_barrier_sync_invoke_and_complete + 172 7 TwilioVideo 0x0000000102a0c010 -[TVICameraCapturePipeline applicationWillEnterForeground] + 116 8 CoreFoundation 0x00000001d64c73f0 5CDC5D9A-E506-3740-B64E-BB30867B4F1B + 226288 9 CoreFoundation 0x00000001d656e474 5CDC5D9A-E506-3740-B64E-BB30867B4F1B + 910452 10 CoreFoundation 0x00000001d6551724 5CDC5D9A-E506-3740-B64E-BB30867B4F1B + 792356 11 CoreFoundation 0x00000001d64dba08 _CFXNotificationPost + 704 12 Foundation 0x00000001d08d8ffc C431ACB6-FE04-3D28-B677-4DE6E1C7D81F + 380924 13 UIKitCore 0x00000001d89d2f60 179501B6-0FC2-344A-B969-B4E3961EBE10 + 3694432 14 UIKitCore 0x00000001d8cf0208 179501B6-0FC2-344A-B969-B4E3961EBE10 + 6959624 15 UIKitCore 0x00000001d86eb600 _UIScenePerformActionsWithLifecycleActionMask + 108 16 UIKitCore 0x00000001d8cf00a8 179501B6-0FC2-344A-B969-B4E3961EBE10 + 6959272 17 UIKitCore 0x00000001d898daa4 179501B6-0FC2-344A-B969-B4E3961EBE10 + 3410596 18 UIKitCore 0x00000001d898d628 179501B6-0FC2-344A-B969-B4E3961EBE10 + 3409448 19 UIKitCore 0x00000001d898d260 179501B6-0FC2-344A-B969-B4E3961EBE10 + 3408480 20 UIKitCore 0x00000001d88dd828 179501B6-0FC2-344A-B969-B4E3961EBE10 + 2689064 21 UIKitCore 0x00000001d8fbffa8 179501B6-0FC2-344A-B969-B4E3961EBE10 + 9908136 22 UIKitCore 0x00000001d9058f98 _UISceneSettingsDiffActionPerformChangesWithTransitionContextAndCompletion + 224 23 UIKitCore 0x00000001d8789958 179501B6-0FC2-344A-B969-B4E3961EBE10 + 1296728 24 UIKitCore 0x00000001d8bfc7a8 179501B6-0FC2-344A-B969-B4E3961EBE10 + 5961640 25 UIKitCore 0x00000001d885c0b8 179501B6-0FC2-344A-B969-B4E3961EBE10 + 2158776 26 UIKitCore 0x00000001d885bf28 179501B6-0FC2-344A-B969-B4E3961EBE10 + 2158376 27 UIKitCore 0x00000001d885bd68 179501B6-0FC2-344A-B969-B4E3961EBE10 + 2157928 28 FrontBoardServices 0x00000001ec0b5cdc 65E92D01-137F-3E16-98A6-A33706699695 + 15580 29 FrontBoardServices 0x00000001ec0b5b18 65E92D01-137F-3E16-98A6-A33706699695 + 15128 30 FrontBoardServices 0x00000001ec0b9294 65E92D01-137F-3E16-98A6-A33706699695 + 29332 31 FrontBoardServices 0x00000001ec0b91b0 65E92D01-137F-3E16-98A6-A33706699695 + 29104 32 libdispatch.dylib 0x00000001013f605c _dispatch_client_callout + 20 33 libdispatch.dylib 0x00000001013f9ad8 _dispatch_block_invoke_direct + 368 34 FrontBoardServices 0x00000001ec0c33b0 65E92D01-137F-3E16-98A6-A33706699695 + 70576 35 FrontBoardServices 0x00000001ec0c2f4c 65E92D01-137F-3E16-98A6-A33706699695 + 69452 36 FrontBoardServices 0x00000001ec0c572c 65E92D01-137F-3E16-98A6-A33706699695 + 79660 37 CoreFoundation 0x00000001d6565f54 5CDC5D9A-E506-3740-B64E-BB30867B4F1B + 876372 38 CoreFoundation 0x00000001d657232c 5CDC5D9A-E506-3740-B64E-BB30867B4F1B + 926508 39 CoreFoundation 0x00000001d64f6210 5CDC5D9A-E506-3740-B64E-BB30867B4F1B + 418320 40 CoreFoundation 0x00000001d650bba8 5CDC5D9A-E506-3740-B64E-BB30867B4F1B + 506792 41 CoreFoundation 0x00000001d6510ed4 CFRunLoopRunSpecific + 612 42 GraphicsServices 0x000000020f80e368 GSEventRunModal + 164 43 UIKitCore 0x00000001d89ef3d0 179501B6-0FC2-344A-B969-B4E3961EBE10 + 3810256 44 UIKitCore 0x00000001d89ef034 UIApplicationMain + 340 45 SwiftUI 0x00000001d9ffc014 494A7D57-90AF-3E90-9623-7038275B8D87 + 1904660 46 SwiftUI 0x00000001d9f5d16c 494A7D57-90AF-3E90-9623-7038275B8D87 + 1253740 47 SwiftUI 0x00000001d9f464bc 494A7D57-90AF-3E90-9623-7038275B8D87 + 1160380 48 SharedUITest 0x000000010079e608 $s12SharedUITest0aB3AppV5$mainyyFZ + 40 49 SharedUITest 0x000000010079e6b0 main + 12 50 dyld 0x00000001f4b78960 CB3FF411-4762-34D2-86A4-ECA13F9FB6C3 + 88416 2022-11-22 09:46:45.464784+0000 SharedUITest[950:128236] DEBUG:Twilio:[Platform](Thread 0x0x101a4f4e0): Preparing renderer for frame size: {640, 480}, view bounds: {{0, 0}, {414, 444}} 2022-11-22 09:46:45.466367+0000 SharedUITest[950:128236] DEBUG:Twilio:[Platform](Thread 0x0x101a4f4e0): Preparing renderer for frame size: {640, 480}, view bounds: {{0, 0}, {414, 444}} 2022-11-22 09:46:45.788633+0000 SharedUITest[950:129728] INFO:Twilio:Platform: AVCaptureSession interruption ended 2022-11-22 09:46:45.789369+0000 SharedUITest[950:129728] INFO:Twilio:Platform: Best device format was: <AVCaptureDeviceFormat: 0x2817722e0 'vide'/'420f' 640x 480, { 1- 60 fps}, photo dims:{640x480,4032x3024}, fov:73.292, max zoom:189.00 (upscales @6.30), ISO:23.0-2208.0, SS:0.000021-1.000000, supports wide color, supports Portrait Effect> 2022-11-22 09:46:45.789785+0000 SharedUITest[950:128587] DEBUG:Twilio:Platform: Video device will start running: <AVCaptureFigVideoDevice: 0x105930000 [Front Camera][com.apple.avfoundation.avcapturedevice.built-in_video:1]>. 2022-11-22 09:46:45.791102+0000 SharedUITest[950:129728] INFO:Twilio:Platform: AVCaptureSession started running 2022-11-22 09:46:46.108532+0000 SharedUITest[950:128236] DEBUG:Twilio:[Platform](Thread 0x0x101a4f4e0): Preparing renderer for frame size: {640, 480}, view bounds: {{0, 0}, {414, 444}} 2022-11-22 09:46:46.109200+0000 SharedUITest[950:128236] DEBUG:Twilio:[Platform](Thread 0x0x101a4f4e0): Preparing renderer for frame size: {640, 480}, view bounds: {{0, 0}, {414, 444}} 2022-11-22 09:46:47.315829+0000 SharedUITest[950:128881] DEBUG:Twilio:Core: <0x282a56b20> TcmpWssTransport::send (20 bytes) {"type":"heartbeat"} 2022-11-22 09:46:48.248915+0000 SharedUITest[950:128881] DEBUG:Twilio:Core: <0x282a56b20> TcmpWssTransport::onMessage (20 bytes) {"type":"heartbeat"} 2022-11-22 09:46:52.116876+0000 SharedUITest[950:128881] DEBUG:Twilio:Core: <0x282a56b20> TcmpWssTransport::send (20 bytes) {"type":"heartbeat"} 2022-11-22 09:46:52.347703+0000 SharedUITest[950:128881] DEBUG:Twilio:Core: <0x282a56b20> TcmpWssTransport::onMessage (20 bytes) {"type":"heartbeat"} 2022-11-22 09:46:56.448506+0000 SharedUITest[950:128881] DEBUG:Twilio:Core: <0x282a56b20> TcmpWssTransport::onMessage (20 bytes) {"type":"heartbeat"} 2022-11-22 09:46:56.917892+0000 SharedUITest[950:128881] DEBUG:Twilio:Core: <0x282a56b20> TcmpWssTransport::send (20 bytes) {"type":"heartbeat"} 2022-11-22 09:47:00.451758+0000 SharedUITest[950:128881] DEBUG:Twilio:Core: <0x282a56b20> TcmpWssTransport::onMessage (20 bytes) {"type":"heartbeat"} 2022-11-22 09:47:01.718886+0000 SharedUITest[950:128881] DEBUG:Twilio:Core: <0x282a56b20> TcmpWssTransport::send (20 bytes) {"type":"heartbeat"} 2022-11-22 09:47:04.450012+0000 SharedUITest[950:128881] DEBUG:Twilio:Core: <0x282a56b20> TcmpWssTransport::onMessage (20 bytes) {"type":"heartbeat"} 2022-11-22 09:47:06.519275+0000 SharedUITest[950:128881] DEBUG:Twilio:Core: <0x282a56b20> TcmpWssTransport::send (20 bytes) {"type":"heartbeat"} 2022-11-22 09:47:07.501286+0000 SharedUITest[950:128236] DEBUG:Twilio:[Core](Thread 0x0x101a4f4e0): API Call disconnect 2022-11-22 09:47:07.501386+0000 SharedUITest[950:128236] DEBUG:Twilio:[Core](Thread 0x0x101a4f4e0): RoomSignalingImpl: State transition successful: kConnected -> kDisconnecting 2022-11-22 09:47:07.501497+0000 SharedUITest[950:128905] DEBUG:Twilio:Core: Disconnecting from RMb396250f7cf1df2118238de9ad29a2dd. 2022-11-22 09:47:07.501603+0000 SharedUITest[950:128905] DEBUG:Twilio:Core: RemoteParticipantSignaling::~RemoteParticipantSignaling: participant_sid: PA10b4ffdcbcb81393f727fed475734920 2022-11-22 09:47:07.501667+0000 SharedUITest[950:128905] DEBUG:Twilio:Core: <0x11dc04af0> InsightsPublisher::stop



### Versions

All relevant version information for the issue.

#### Video iOS SDK

5.2.1 via SPM

#### Xcode

Version 14.1 (14B47b)

#### iOS Version

16.1

#### iOS Device

iPhone SE / iPhone 14 Pro
Dinalli commented 1 year ago

Upgraded to Version 5.3.0 and the same issue occurs there too.

Dinalli commented 1 year ago

Oaky some further info on this now and the fact it now breaks our current App -

Whilst running a video session if an external phone call comes in whilst in a Twilio video call, the video freezes and the UI does not respond.

Extra stack trace below.

AppEnteredForeground CALL CONNECTED Thread Performance Checker: -[AVCaptureSession startRunning] should be called from background thread. Calling it on the main thread can lead to UI unresponsiveness PID: 6987, TID: 2277767 Backtrace

3 TwilioVideo 0x0000000103093008 -[TVICameraCapturePipeline startSession] + 52 4 TwilioVideo 0x0000000103095fdc __58-[TVICameraCapturePipeline applicationWillEnterForeground]_block_invoke + 124 5 libdispatch.dylib 0x0000000101ada05c _dispatch_client_callout + 20 6 libdispatch.dylib 0x0000000101aeb884 _dispatch_lane_barrier_sync_invoke_and_complete + 172 7 TwilioVideo 0x0000000103095f50 -[TVICameraCapturePipeline applicationWillEnterForeground] + 116 8 CoreFoundation 0x00000001baa8b3b0 159AFDDD-C8E6-3C80-88DF-9D7BD2D9F0DA + 226224 9 CoreFoundation 0x00000001bab32454 159AFDDD-C8E6-3C80-88DF-9D7BD2D9F0DA + 910420 10 CoreFoundation 0x00000001bab15704 159AFDDD-C8E6-3C80-88DF-9D7BD2D9F0DA + 792324 11 CoreFoundation 0x00000001baa9f9c8 _CFXNotificationPost + 704 12 Foundation 0x00000001b4e2fd9c 02446407-7BE3-392C-9B07-503C8B8D85CD + 380316 13 UIKitCore 0x00000001bcfba3fc 654FF2F0-015F-369C-84B2-27608D64E637 + 3691516 14 UIKitCore 0x00000001bd2d7df4 654FF2F0-015F-369C-84B2-27608D64E637 + 6958580 15 UIKitCore 0x00000001bccd320c _UIScenePerformActionsWithLifecycleActionMask + 108

piyushtank commented 1 year ago

@Dinalli Thanks for reaching out. We are working on the issue and plan to ship the fix soon.

piyushtank commented 1 year ago

@Dinalli We have shipped 5.5.0 with the fix. Thanks.