_os_unfair_lock_recursive_abort in [ADBMediaHeartbeat playheadInfo] and [ADBMediaHeartbeat qosInfo]

[nuno-vieira]

This crash is one of the most serious we have currently on our application since we enabled Adobe's Heartbeat. It is affecting 12k users currently, even tho that is still a small % percentage of our user base. Currently this is our most priority bug.

We get crash on either from qosInfo or playHeadInfo methods. And it seems it something related with accessing deallocated objects. Would be nice to understand what these functions are doing with the data we provide since we are running the SDK in Kotlin/Native, and the concurrency here is a bit different than native iOS, and we need to be careful when mutating data in different threads. Please let me know if you need more information, and do provide us the information about what these methods are doing, if that is possible at all.

qosInfo stack trace:

Crashed: com.apple.main-thread
EXC_BREAKPOINT 0x00000002378e6024

0  libsystem_platform.dylib       0x2378e6024 _os_unfair_lock_recursive_abort + 36
1  libsystem_platform.dylib       0x2378e737c _os_unfair_lock_lock_slow + 272
2  libobjc.A.dylib                0x236ecc270 objc_object::sidetable_clearDeallocating() + 104
3  libobjc.A.dylib                0x236ec1700 objc_destructInstance + 140
4  libswiftCore.dylib             0x26530ecd4 swift_deallocClassInstance + 32
5  libswiftCore.dylib             0x26530ec68 _swift_release_dealloc + 28
6  AddonManager                   0x102049900 (anonymous namespace)::freeContainer(ContainerHeader*) + 316
7  AddonManager                   0x10204817c (anonymous namespace)::garbageCollect(MemoryState*, bool) + 512
8  AddonManager                   0x10204d364 ReleaseHeapRefStrict + 208
9  AddonManager                   0x10204ae3c BackRefFromAssociatedObject::tryAddRef() + 580
10 AddonManager                   0x102044f0c (anonymous namespace)::_tryRetainImp(objc_object*, objc_selector*) + 36
11 libobjc.A.dylib                0x236ecb7cc objc_loadWeakRetained + 352
12 AddonManager                   0x1020bb4b0 -[ADBMediaHeartbeat qosInfo] + 60
13 AddonManager                   0x102089784 -[ADB_VHB_MediaHeartbeatVideoPlayerPluginDelegate getQoSInfo] + 48
14 AddonManager                   0x1020d1038 -[ADB_VHB_VideoPlayerPlugin cacheQoSInfo] + 112
15 AddonManager                   0x1020d04f0 __46-[ADB_VHB_VideoPlayerPlugin setupDataResolver]_block_invoke559 + 52
16 AddonManager                   0x1020a940c -[ADB_HB_Callback call:] + 76
17 AddonManager                   0x1020d077c __46-[ADB_VHB_VideoPlayerPlugin setupDataResolver]_block_invoke567 + 320
18 AddonManager                   0x1020a940c -[ADB_HB_Callback call:] + 76
19 AddonManager                   0x1020a8bc8 -[ADB_HB_BasePlugin resolveData:] + 200
20 AddonManager                   0x1020d2df4 -[ADB_HB_PluginManager request:keys:] + 184
21 AddonManager                   0x1020d2858 -[ADB_HB_PluginManager triggerEvent:] + 1080
22 AddonManager                   0x1020a9110 -[ADB_HB_BasePlugin _trigger:info:] + 288
23 AddonManager                   0x1020b129c -[ADBMediaHeartbeat _cmdBufferStart:] + 40
24 AddonManager                   0x1020b9504 __32-[ADBMediaHeartbeat _setupRules]_block_invoke643 + 56
25 AddonManager                   0x1020a940c -[ADB_HB_Callback call:] + 76
26 AddonManager                   0x1020a0ef8 -[ADBRuleEngine processRule:ruleEngineContext:] + 716
27 AddonManager                   0x1020b2420 -[ADBMediaHeartbeat processRule:ruleEngineContext:] + 92
28 AddonManager                   0x1020af13c -[ADBMediaHeartbeat trackEvent:mediaObject:data:] + 1260
29 AddonManager                   0x101e0efb8 kfun:com.sky.core.player.sdk.addon.adobe.AdobeMediaHeartbeatWrapperImpl.trackEvent#internal + 210 (AdobeMediaHeartbeatWrapperImpl.kt:210)
30 AddonManager                   0x101e0d4fc kfun:com.sky.core.player.sdk.addon.adobe.AdobeMediaHeartbeatWrapperImpl#trackDidStartBuffering(){} + 80 (AdobeMediaHeartbeatWrapperImpl.kt:80)
31 AddonManager                   0x101e49348 kfun:com.sky.core.player.sdk.addon.adobe.AdobeMediaAddon#nativePlayerIsBuffering(){} + 195 (AdobeMediaAddon.kt:195)
32 AddonManager                   0x101e3ecb0 kfun:com.sky.core.player.sdk.addon.AddonManagerImpl.$nativePlayerIsBuffering$lambda-22$FUNCTION_REFERENCE$35.$<bridge-UNNN>invoke(-1:0){}#internal + 298 (AddonManagerImpl.kt:298)
33 AddonManager                   0x101e39380 kfun:com.sky.core.player.sdk.addon.AddonManagerImpl.forEachAddon#internal + 32 (Iterators.kt:32)
34 AddonManager                   0x101e36870 kfun:com.sky.core.player.sdk.addon.AddonManagerImpl#nativePlayerIsBuffering(){} + 298 (AddonManagerImpl.kt:298)
35 AddonManager                   0x101f6c870 objc2kotlin.1413 + 1 (<compiler-generated>:1)
36 libdispatch.dylib              0x237717a38 _dispatch_call_block_and_release + 24
37 libdispatch.dylib              0x2377187d4 _dispatch_client_callout + 16
38 libdispatch.dylib              0x2376c6008 _dispatch_main_queue_callback_4CF$VARIANT$mp + 1068
39 CoreFoundation                 0x237c6bb20 __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__ + 12
40 CoreFoundation                 0x237c66a58 __CFRunLoopRun + 1924
41 CoreFoundation                 0x237c65fb4 CFRunLoopRunSpecific + 436
42 GraphicsServices               0x239e6779c GSEventRunModal + 104
43 UIKitCore                      0x263f43c38 UIApplicationMain + 212
44 PeacockMobile-US               0x1008af340 main + 4309463872 (<compiler-generated>:4309463872)
45 libdyld.dylib                  0x2377298e0 start + 4

playHeadInfo stack trace:

Crashed: com.apple.main-thread
EXC_BREAKPOINT 0x00000002378e6024

0  libsystem_platform.dylib       0x1cdc371e4 _os_unfair_lock_recursive_abort + 36
1  libsystem_platform.dylib       0x1cdc35154 _os_unfair_lock_lock_slow + 304
2  libobjc.A.dylib                0x19a77b0bc objc_object::sidetable_clearDeallocating() + 100
3  libobjc.A.dylib                0x19a771d74 objc_destructInstance + 140
4  libsystem_blocks.dylib         0x1cdbd07a4 _Block_release + 216
5  AddonManager                   0x106275900 (anonymous namespace)::freeContainer(ContainerHeader*) + 316
6  AddonManager                   0x10627417c (anonymous namespace)::garbageCollect(MemoryState*, bool) + 512
7  AddonManager                   0x106279364 ReleaseHeapRefStrict + 208
8  AddonManager                   0x106276e3c BackRefFromAssociatedObject::tryAddRef() + 580
9  AddonManager                   0x106270f0c (anonymous namespace)::_tryRetainImp(objc_object*, objc_selector*) + 36
10 libobjc.A.dylib                0x19a77acdc objc_loadWeakRetained + 340
11 AddonManager                   0x1062e7424 -[ADBMediaHeartbeat playheadInfo] + 64
12 AddonManager                   0x1062b57d8 -[ADB_VHB_MediaHeartbeatVideoPlayerPluginDelegate getPlayheadInfo] + 48
13 AddonManager                   0x1062fd19c -[ADB_VHB_VideoPlayerPlugin cachePlayheadInfo] + 112
14 AddonManager                   0x1062fa55c __46-[ADB_VHB_VideoPlayerPlugin setupDataResolver]_block_invoke353 + 52
15 AddonManager                   0x1062d540c -[ADB_HB_Callback call:] + 76
16 AddonManager                   0x1062fc77c __46-[ADB_VHB_VideoPlayerPlugin setupDataResolver]_block_invoke567 + 320
17 AddonManager                   0x1062d540c -[ADB_HB_Callback call:] + 76
18 AddonManager                   0x1062d4bc8 -[ADB_HB_BasePlugin resolveData:] + 200
19 AddonManager                   0x1062fedf4 -[ADB_HB_PluginManager request:keys:] + 184
20 AddonManager                   0x1062fe858 -[ADB_HB_PluginManager triggerEvent:] + 1080
21 AddonManager                   0x1062d5110 -[ADB_HB_BasePlugin _trigger:info:] + 288
22 AddonManager                   0x1062f6ea0 -[ADB_VHB_VideoPlayerPlugin trackPlay] + 224
23 AddonManager                   0x1062dd34c -[ADBMediaHeartbeat _cmdPlay:] + 40
24 AddonManager                   0x1062e4f7c __32-[ADBMediaHeartbeat _setupRules]_block_invoke585 + 56
25 AddonManager                   0x1062d540c -[ADB_HB_Callback call:] + 76
26 AddonManager                   0x1062ccef8 -[ADBRuleEngine processRule:ruleEngineContext:] + 716
27 AddonManager                   0x1062de420 -[ADBMediaHeartbeat processRule:ruleEngineContext:] + 92
28 AddonManager                   0x1060390a4 kfun:com.sky.core.player.sdk.addon.adobe.AdobeMediaHeartbeatWrapperImpl#trackPlay(){} + 55 (AdobeMediaHeartbeatWrapperImpl.kt:55)
29 AddonManager                   0x106074fd8 kfun:com.sky.core.player.sdk.addon.adobe.AdobeMediaAddon#nativePlayerWillPlay(){} + 185 (AdobeMediaAddon.kt:185)
30 AddonManager                   0x10606a9c0 kfun:com.sky.core.player.sdk.addon.AddonManagerImpl.$nativePlayerWillPlay$lambda-19$FUNCTION_REFERENCE$32.$<bridge-UNNN>invoke(-1:0){}#internal + 289 (AddonManagerImpl.kt:289)
31 AddonManager                   0x106065380 kfun:com.sky.core.player.sdk.addon.AddonManagerImpl.forEachAddon#internal + 32 (Iterators.kt:32)
32 AddonManager                   0x1060625e8 kfun:com.sky.core.player.sdk.addon.AddonManagerImpl#nativePlayerWillPlay(){} + 289 (AddonManagerImpl.kt:289)
33 AddonManager                   0x106198b2c objc2kotlin.1416 + 1 (<compiler-generated>:1)
34 libdispatch.dylib              0x1862defd0 _dispatch_call_block_and_release + 32
35 libdispatch.dylib              0x1862e0ac8 _dispatch_client_callout + 20
36 libdispatch.dylib              0x1862ee04c _dispatch_main_queue_callback_4CF + 836
37 CoreFoundation                 0x18665f1e4 __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__ + 16
38 CoreFoundation                 0x1866593b4 __CFRunLoopRun + 2508
39 CoreFoundation                 0x1866584bc CFRunLoopRunSpecific + 600
40 GraphicsServices               0x19d0dd820 GSEventRunModal + 164
41 UIKitCore                      0x188ffc734 -[UIApplication _run] + 1072
42 UIKitCore                      0x189001e10 UIApplicationMain + 168
43 PeacockMobile-US               0x1049bb340 main + 4311462720 (<compiler-generated>:4311462720)
44 libdyld.dylib                  0x18631fe60 start + 4

Full Stack Trace of one of the examples above:

Crashed: com.apple.main-thread
EXC_BREAKPOINT 0x00000001cdc371e4

Crashed: com.apple.main-thread
0  libsystem_platform.dylib       0x1cdc371e4 _os_unfair_lock_recursive_abort + 36
1  libsystem_platform.dylib       0x1cdc35154 _os_unfair_lock_lock_slow + 304
2  libobjc.A.dylib                0x19a77b0bc objc_object::sidetable_clearDeallocating() + 100
3  libobjc.A.dylib                0x19a771d74 objc_destructInstance + 140
4  libsystem_blocks.dylib         0x1cdbd07a4 _Block_release + 216
5  AddonManager                   0x106275900 (anonymous namespace)::freeContainer(ContainerHeader*) + 316
6  AddonManager                   0x10627417c (anonymous namespace)::garbageCollect(MemoryState*, bool) + 512
7  AddonManager                   0x106279364 ReleaseHeapRefStrict + 208
8  AddonManager                   0x106276e3c BackRefFromAssociatedObject::tryAddRef() + 580
9  AddonManager                   0x106270f0c (anonymous namespace)::_tryRetainImp(objc_object*, objc_selector*) + 36
10 libobjc.A.dylib                0x19a77acdc objc_loadWeakRetained + 340
11 AddonManager                   0x1062e7424 -[ADBMediaHeartbeat playheadInfo] + 64
12 AddonManager                   0x1062b57d8 -[ADB_VHB_MediaHeartbeatVideoPlayerPluginDelegate getPlayheadInfo] + 48
13 AddonManager                   0x1062fd19c -[ADB_VHB_VideoPlayerPlugin cachePlayheadInfo] + 112
14 AddonManager                   0x1062fa55c __46-[ADB_VHB_VideoPlayerPlugin setupDataResolver]_block_invoke353 + 52
15 AddonManager                   0x1062d540c -[ADB_HB_Callback call:] + 76
16 AddonManager                   0x1062fc77c __46-[ADB_VHB_VideoPlayerPlugin setupDataResolver]_block_invoke567 + 320
17 AddonManager                   0x1062d540c -[ADB_HB_Callback call:] + 76
18 AddonManager                   0x1062d4bc8 -[ADB_HB_BasePlugin resolveData:] + 200
19 AddonManager                   0x1062fedf4 -[ADB_HB_PluginManager request:keys:] + 184
20 AddonManager                   0x1062fe858 -[ADB_HB_PluginManager triggerEvent:] + 1080
21 AddonManager                   0x1062d5110 -[ADB_HB_BasePlugin _trigger:info:] + 288
22 AddonManager                   0x1062f6ea0 -[ADB_VHB_VideoPlayerPlugin trackPlay] + 224
23 AddonManager                   0x1062dd34c -[ADBMediaHeartbeat _cmdPlay:] + 40
24 AddonManager                   0x1062e4f7c __32-[ADBMediaHeartbeat _setupRules]_block_invoke585 + 56
25 AddonManager                   0x1062d540c -[ADB_HB_Callback call:] + 76
26 AddonManager                   0x1062ccef8 -[ADBRuleEngine processRule:ruleEngineContext:] + 716
27 AddonManager                   0x1062de420 -[ADBMediaHeartbeat processRule:ruleEngineContext:] + 92
28 AddonManager                   0x1060390a4 kfun:com.sky.core.player.sdk.addon.adobe.AdobeMediaHeartbeatWrapperImpl#trackPlay(){} + 55 (AdobeMediaHeartbeatWrapperImpl.kt:55)
29 AddonManager                   0x106074fd8 kfun:com.sky.core.player.sdk.addon.adobe.AdobeMediaAddon#nativePlayerWillPlay(){} + 185 (AdobeMediaAddon.kt:185)
30 AddonManager                   0x10606a9c0 kfun:com.sky.core.player.sdk.addon.AddonManagerImpl.$nativePlayerWillPlay$lambda-19$FUNCTION_REFERENCE$32.$<bridge-UNNN>invoke(-1:0){}#internal + 289 (AddonManagerImpl.kt:289)
31 AddonManager                   0x106065380 kfun:com.sky.core.player.sdk.addon.AddonManagerImpl.forEachAddon#internal + 32 (Iterators.kt:32)
32 AddonManager                   0x1060625e8 kfun:com.sky.core.player.sdk.addon.AddonManagerImpl#nativePlayerWillPlay(){} + 289 (AddonManagerImpl.kt:289)
33 AddonManager                   0x106198b2c objc2kotlin.1416 + 1 (<compiler-generated>:1)
34 libdispatch.dylib              0x1862defd0 _dispatch_call_block_and_release + 32
35 libdispatch.dylib              0x1862e0ac8 _dispatch_client_callout + 20
36 libdispatch.dylib              0x1862ee04c _dispatch_main_queue_callback_4CF + 836
37 CoreFoundation                 0x18665f1e4 __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__ + 16
38 CoreFoundation                 0x1866593b4 __CFRunLoopRun + 2508
39 CoreFoundation                 0x1866584bc CFRunLoopRunSpecific + 600
40 GraphicsServices               0x19d0dd820 GSEventRunModal + 164
41 UIKitCore                      0x188ffc734 -[UIApplication _run] + 1072
42 UIKitCore                      0x189001e10 UIApplicationMain + 168
43 AppName               0x1049bb340 main + 4311462720 (<compiler-generated>:4311462720)
44 libdyld.dylib                  0x18631fe60 start + 4com.apple.uikit.eventfetch-thread

com.apple.uikit.eventfetch-thread
0  libsystem_kernel.dylib         0x1b25efdd0 mach_msg_trap + 8
1  libsystem_kernel.dylib         0x1b25ef184 mach_msg + 76
2  CoreFoundation                 0x18665ecf8 __CFRunLoopServiceMachPort + 380
3  CoreFoundation                 0x186658ea8 __CFRunLoopRun + 1216
4  CoreFoundation                 0x1866584bc CFRunLoopRunSpecific + 600
5  Foundation                     0x1878d5e30 -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 232
6  Foundation                     0x1878d5d08 -[NSRunLoop(NSRunLoop) runUntilDate:] + 92
7  UIKitCore                      0x1890ace94 -[UIEventFetcher threadMain] + 516
8  Foundation                     0x187a400f0 __NSThread__start__ + 864
9  libsystem_pthread.dylib        0x1cdc3aca8 _pthread_start + 320
10 libsystem_pthread.dylib        0x1cdc43788 thread_start + 8com.google.firebase.crashlytics.MachExceptionServer

com.google.firebase.crashlytics.MachExceptionServer
0  AppName               0x104d91c1c FIRCLSProcessRecordAllThreads + 392 (FIRCLSProcess.c:392)
1   AppName               0x104d92000 FIRCLSProcessRecordAllThreads + 423 (FIRCLSProcess.c:423)
2  AppName               0x104d88914 FIRCLSHandler + 34 (FIRCLSHandler.m:34)
3  AppName              0x104d8b250 FIRCLSMachExceptionServer + 524 (FIRCLSMachException.c:524)
4  libsystem_pthread.dylib        0x1cdc3aca8 _pthread_start + 320
5  libsystem_pthread.dylib        0x1cdc43788 thread_start + 8com.facebook.react.JavaScript

com.facebook.react.JavaScript
0  libsystem_kernel.dylib         0x1b26143d0 __ulock_wait + 8
1  libsystem_platform.dylib       0x1cdc350e8 _os_unfair_lock_lock_slow + 196
2  libobjc.A.dylib                0x19a77abf0 objc_loadWeakRetained + 104
3  AppName               0x104e42cac -[RCTModuleData methodQueue] + 352 (RCTModuleData.mm:352)
4  AppName               0x104e33cd0 -[RCTDisplayLink _jsThreadUpdate:] + 120 (RCTDisplayLink.m:120)
5  QuartzCore                     0x1898ec6a4 CA::Display::DisplayLink::dispatch_items(unsigned long long, unsigned long long, unsigned long long) + 664
6  QuartzCore                     0x1899c31a0 display_timer_callback(__CFMachPort*, void*, long, void*) + 280
7  CoreFoundation                 0x18663bce4 __CFMachPortPerform + 176
8  CoreFoundation                 0x186660098 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE1_PERFORM_FUNCTION__ + 60
9  CoreFoundation                 0x18665f440 __CFRunLoopDoSource1 + 596
10 CoreFoundation                 0x186659320 __CFRunLoopRun + 2360
11 CoreFoundation                 0x1866584bc CFRunLoopRunSpecific + 600
12 AppName               0x104e26e00 +[RCTCxxBridge runRunLoop] + 268 (RCTCxxBridge.mm:268)
13 Foundation                     0x187a400f0 __NSThread__start__ + 864
14 libsystem_pthread.dylib        0x1cdc3aca8 _pthread_start + 320
15 libsystem_pthread.dylib        0x1cdc43788 thread_start + 8JavaScriptCore bmalloc scavenger

JavaScriptCore bmalloc scavenger
0  libsystem_kernel.dylib         0x1b261375c __psynch_cvwait + 8
1  libsystem_pthread.dylib        0x1cdc3e550 _pthread_cond_wait + 1192
2  libc++.1.dylib                 0x19a7fac8c std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) + 28
3  JavaScriptCore                 0x190702188 void std::__1::condition_variable_any::wait<std::__1::unique_lock<bmalloc::Mutex> >(std::__1::unique_lock<bmalloc::Mutex>&) + 108
4  JavaScriptCore                 0x190706728 bmalloc::Scavenger::threadRunLoop() + 196
5  JavaScriptCore                 0x190706364 bmalloc::Scavenger::Scavenger(std::__1::scoped_lock<bmalloc::Mutex> const&) + 14
6  JavaScriptCore                 0x190707828 std::__1::__thread_specific_ptr<std::__1::__thread_struct>::set_pointer(std::__1::__thread_struct*) + 46
7  libsystem_pthread.dylib        0x1cdc3aca8 _pthread_start + 320
8  libsystem_pthread.dylib        0x1cdc43788 thread_start + 8com.apple.NSURLConnectionLoader

com.apple.NSURLConnectionLoader
0  libsystem_kernel.dylib         0x1b25efdd0 mach_msg_trap + 8
1  libsystem_kernel.dylib         0x1b25ef184 mach_msg + 76
2  CoreFoundation                 0x18665ecf8 __CFRunLoopServiceMachPort + 380
3  CoreFoundation                 0x186658ea8 __CFRunLoopRun + 1216
4  CoreFoundation                 0x1866584bc CFRunLoopRunSpecific + 600
5  CFNetwork                      0x186f11050 _CFURLStorageSessionCopyCache + 63240
6  Foundation                     0x187a400f0 __NSThread__start__ + 864
7  libsystem_pthread.dylib        0x1cdc3aca8 _pthread_start + 320
8  libsystem_pthread.dylib        0x1cdc43788 thread_start + 8com.apple.CoreMotion.MotionThread

com.apple.CoreMotion.MotionThread
0  libsystem_kernel.dylib         0x1b25efdd0 mach_msg_trap + 8
1  libsystem_kernel.dylib         0x1b25ef184 mach_msg + 76
2  CoreFoundation                 0x18665ecf8 __CFRunLoopServiceMachPort + 380
3  CoreFoundation                 0x186658ea8 __CFRunLoopRun + 1216
4  CoreFoundation                 0x1866584bc CFRunLoopRunSpecific + 600
5  CoreFoundation                 0x186659590 CFRunLoopRun + 64
6  CoreMotion                     0x191401d50 CLClientCreateIso6709Notation + 177744
7  libsystem_pthread.dylib        0x1cdc3aca8 _pthread_start + 320
8  libsystem_pthread.dylib        0x1cdc43788 thread_start + 8AVAudioSession Notify Thread

AVAudioSession Notify Thread
0  libsystem_kernel.dylib         0x1b25efdd0 mach_msg_trap + 8
1  libsystem_kernel.dylib         0x1b25ef184 mach_msg + 76
2  CoreFoundation                 0x18665ecf8 __CFRunLoopServiceMachPort + 380
3  CoreFoundation                 0x186658ea8 __CFRunLoopRun + 1216
4  CoreFoundation                 0x1866584bc CFRunLoopRunSpecific + 600
5  AudioSession                   0x18defa2ac GenericRunLoopThread::Entry(void*) + 164
6  AudioSession                   0x18defc390 CAPThread::Entry(CAPThread*) + 92
7  libsystem_pthread.dylib        0x1cdc3aca8 _pthread_start + 320
8  libsystem_pthread.dylib        0x1cdc43788 thread_start + 8ComScore-Thread

ComScore-Thread
0  libsystem_kernel.dylib         0x1b261375c __psynch_cvwait + 8
1  libsystem_pthread.dylib        0x1cdc3e57c _pthread_cond_wait + 1236
2  AddonManager                   0x1063b1824 ComScore::WaitableEvent::wait(int) const + 188
3  AddonManager                   0x1063fce7c ComScore::ContextTasksThread<ComScore::Core>::run() + 224
4  AddonManager                   0x1063b1ccc ComScore::Thread::threadEntryPoint() + 104
5  AddonManager                   0x1063b3270 threadEntryProc + 32
6  libsystem_pthread.dylib        0x1cdc3aca8 _pthread_start + 320
7  libsystem_pthread.dylib        0x1cdc43788 thread_start + 8ComScore-NetworkThread

ComScore-NetworkThread
0  libsystem_kernel.dylib         0x1b261375c __psynch_cvwait + 8
1  libsystem_pthread.dylib        0x1cdc3e57c _pthread_cond_wait + 1236
2  AddonManager                   0x1063b1824 ComScore::WaitableEvent::wait(int) const + 188
3  AddonManager                   0x1063fce7c ComScore::ContextTasksThread<ComScore::Core>::run() + 224
4  AddonManager                   0x1063b1ccc ComScore::Thread::threadEntryPoint() + 104
5  AddonManager                   0x1063b3270 threadEntryProc + 32
6  libsystem_pthread.dylib        0x1cdc3aca8 _pthread_start + 320
7  libsystem_pthread.dylib        0x1cdc43788 thread_start + 8com.conviva.ConvivaUtils.cisSharedQueue

com.conviva.ConvivaUtils.cisSharedQueue
0  libsystem_kernel.dylib         0x1b26143d0 __ulock_wait + 8
1  libsystem_platform.dylib       0x1cdc350e8 _os_unfair_lock_lock_slow + 196
2  libobjc.A.dylib                0x19a77abf0 objc_loadWeakRetained + 104
3  AddonManager                   0x106331b90 -[CISTime cisTimeProtocol] + 13 (CISTime.m:13)
4  AddonManager                   0x106331af4 -[CISTime current] + 29 (CISTime.m:29)
5  AddonManager                   0x10631d230 -[CISMonitor sessionTimeMs] + 354 (CISMonitor.m:354)
6  AddonManager                   0x10631cce4 __29-[CISMonitor setPlayerState:]_block_invoke + 303 (CISMonitor.m:303)
7  libdispatch.dylib              0x1862defd0 _dispatch_call_block_and_release + 32
8  libdispatch.dylib              0x1862e0ac8 _dispatch_client_callout + 20
9  libdispatch.dylib              0x1862e7c08 _dispatch_lane_serial_drain + 580
10 libdispatch.dylib              0x1862e8734 _dispatch_lane_invoke + 408
11 libdispatch.dylib              0x1862f2528 _dispatch_workloop_worker_thread + 708
12 libsystem_pthread.dylib        0x1cdc3c908 _pthread_wqthread + 276
13 libsystem_pthread.dylib        0x1cdc4377c start_wqthread + 8com.bskyb.sky.throttler

com.bskyb.sky.throttler
0  libsystem_kernel.dylib         0x1b26143d0 __ulock_wait + 8
1  libsystem_platform.dylib       0x1cdc350e8 _os_unfair_lock_lock_slow + 196
2  libobjc.A.dylib                0x19a77ab04 objc_destroyWeak + 96
3  AppName               0x104cd277c -[SPSMessageSender .cxx_destruct] + 26 (SPSMessageSender.m:26)
4  libobjc.A.dylib                0x19a75b324 object_cxxDestructFromClass(objc_object*, objc_class*) + 112
5  libobjc.A.dylib                0x19a771d38 objc_destructInstance + 80
6  libobjc.A.dylib                0x19a77899c _objc_rootDealloc + 68
7  AppName               0x104d4ac84 -[SPSSignedRequestStrategy .cxx_destruct] + 26 (SPSSignedRequestStrategy.m:26)
8  libobjc.A.dylib                0x19a75b324 object_cxxDestructFromClass(objc_object*, objc_class*) + 112
9  libobjc.A.dylib                0x19a771d38 objc_destructInstance + 80
10 libobjc.A.dylib                0x19a77899c _objc_rootDealloc + 68
11 AppName               0x104d0fcd4 -[SPSRequest .cxx_destruct] + 23 (SPSRequest.m:23)
12 libobjc.A.dylib                0x19a75b324 object_cxxDestructFromClass(objc_object*, objc_class*) + 112
13 libobjc.A.dylib                0x19a771d38 objc_destructInstance + 80
14 libobjc.A.dylib                0x19a77899c _objc_rootDealloc + 68
15 AppName               0x104cdbc50 -[SPSAuthenticatedRequest .cxx_destruct] + 30 (SPSAuthenticatedRequest.m:30)
16 libobjc.A.dylib                0x19a75b324 object_cxxDestructFromClass(objc_object*, objc_class*) + 112
17 libobjc.A.dylib                0x19a771d38 objc_destructInstance + 80
18 libobjc.A.dylib                0x19a77899c _objc_rootDealloc + 68
19 AppName               0x104d2c114 -[SPSStreamingPlayerServicePrivate setRequestSetBookmark:] + 54 (SPSStreamingPlayerServicePrivate.h:54)
20 AppName               0x104ce3144 -[SPSStreamingPlayerServicePrivate setBookmarkForContentId:streamPosition:timestamp:metadata:] + 773 (SPSStreamingPlayerServicePrivate.m:773)
21 AppName               0x104b53ae8 OVPProvider.updateBookmarkForContentId(_:streamPosition:timestamp:metadata:) + 4313135848 (<compiler-generated>:4313135848)
22 AppName               0x104b53a20 protocol witness for OVPApiProvider.updateBookmarkForContentId(_:streamPosition:timestamp:metadata:) in conformance OVPProvider + 4313135648 (<compiler-generated>:4313135648)
23 CoreVideoSDK                   0x106cbfde0 $s12CoreVideoSDK10OVPServiceC21requestBookmarkUpdate11sessionItem14streamPositionyAA07SessionI0O_SdtFySScfU_yycfU_ + 172
24 CoreVideoSDK                   0x106c6561c $s12CoreVideoSDK9ThrottlerC8throttleyyyycFyycfU_ + 164
25 CoreVideoSDK                   0x106c94b30 $sIeg_IeyB_TR + 28
26 libdispatch.dylib              0x1862ee834 _dispatch_block_async_invoke2 + 148
27 libdispatch.dylib              0x1862e0ac8 _dispatch_client_callout + 20
28 libdispatch.dylib              0x1862e7c08 _dispatch_lane_serial_drain + 580
29 libdispatch.dylib              0x1862e8734 _dispatch_lane_invoke + 408
30 libdispatch.dylib              0x1862f2528 _dispatch_workloop_worker_thread + 708
31 libsystem_pthread.dylib        0x1cdc3c908 _pthread_wqthread + 276
32 libsystem_pthread.dylib        0x1cdc4377c start_wqthread + 8com.adobe.analytics.heartbeat.work.network (QOS: UNSPECIFIED)

com.adobe.analytics.heartbeat.work.network (QOS: UNSPECIFIED)
0  libsystem_kernel.dylib         0x1b25efe0c semaphore_wait_trap + 8
1  libdispatch.dylib              0x1862e1018 _dispatch_sema4_wait + 28
2  libdispatch.dylib              0x1862e16b0 _dispatch_semaphore_wait_slow + 132
3  CFNetwork                      0x186d572f4 CFURLConnectionSendSynchronousRequest + 380
4  CFNetwork                      0x186cd1d0c (Missing)
5  AddonManager                   0x1062c37b8 __43-[ADB_HB_Network _onFilterReportAvailable:]_block_invoke + 2144
6  AddonManager                   0x1062d540c -[ADB_HB_Callback call:] + 76
7  AddonManager                   0x1062d54d4 -[ADB_HB_Callback apply:] + 148
8  AddonManager                   0x1062d4604 -[ADB_HB_Command run] + 88
9  AddonManager                   0x1062d70a0 __34-[ADB_HB_CommandQueue addCommand:]_block_invoke33 + 84
10 Foundation                     0x1879fa7f4 __NSBLOCKOPERATION_IS_CALLING_OUT_TO_A_BLOCK__ + 24
11 Foundation                     0x1878ed120 -[NSBlockOperation main] + 104
12 Foundation                     0x1879fcbb8 __NSOPERATION_IS_INVOKING_MAIN__ + 24
13 Foundation                     0x1878ecda8 -[NSOperation start] + 808
14 Foundation                     0x1879fd624 __NSOPERATIONQUEUE_IS_STARTING_AN_OPERATION__ + 24
15 Foundation                     0x1879fd0d0 __NSOQSchedule_f + 184
16 libdispatch.dylib              0x1862ee834 _dispatch_block_async_invoke2 + 148
17 libdispatch.dylib              0x1862e0ac8 _dispatch_client_callout + 20
18 libdispatch.dylib              0x1862e3d60 _dispatch_continuation_pop + 416
19 libdispatch.dylib              0x1862e3494 _dispatch_async_redirect_invoke + 592
20 libdispatch.dylib              0x1862f1104 _dispatch_root_queue_drain + 356
21 libdispatch.dylib              0x1862f18e8 _dispatch_worker_thread2 + 116
22 libsystem_pthread.dylib        0x1cdc3c8cc _pthread_wqthread + 216
23 libsystem_pthread.dylib        0x1cdc4377c start_wqthread + 8Thread #1

Thread
0  libsystem_pthread.dylib        0x1cdc43774 start_wqthread + 6ComScore-StreamingAnalytics-4

ComScore-StreamingAnalytics-4
0  libsystem_kernel.dylib         0x1b261375c __psynch_cvwait + 8
1  libsystem_pthread.dylib        0x1cdc3e57c _pthread_cond_wait + 1236
2  AddonManager                   0x1063b1824 ComScore::WaitableEvent::wait(int) const + 188
3  AddonManager                   0x1064057a8 ComScore::ContextTasksThread<ComScore::StreamingCore>::run() + 224
4  AddonManager                   0x1063b1ccc ComScore::Thread::threadEntryPoint() + 104
5  AddonManager                   0x1063b3270 threadEntryProc + 32
6  libsystem_pthread.dylib        0x1cdc3aca8 _pthread_start + 320
7  libsystem_pthread.dylib        0x1cdc43788 thread_start + 8Thread #2

Thread
0  libsystem_kernel.dylib         0x1b25efe24 semaphore_timedwait_trap + 8
1  libdispatch.dylib              0x1862e1098 _dispatch_sema4_timedwait + 68
2  libdispatch.dylib              0x1862e1678 _dispatch_semaphore_wait_slow + 76
3  libdispatch.dylib              0x1862f0f00 _dispatch_worker_thread + 308
4  libsystem_pthread.dylib        0x1cdc3aca8 _pthread_start + 320
5  libsystem_pthread.dylib        0x1cdc43788 thread_start + 8Thread #3

Thread
0  libsystem_kernel.dylib         0x1b25efe24 semaphore_timedwait_trap + 8
1  libdispatch.dylib              0x1862e1098 _dispatch_sema4_timedwait + 68
2  libdispatch.dylib              0x1862e1678 _dispatch_semaphore_wait_slow + 76
3  libdispatch.dylib              0x1862f0f00 _dispatch_worker_thread + 308
4  libsystem_pthread.dylib        0x1cdc3aca8 _pthread_start + 320
5  libsystem_pthread.dylib        0x1cdc43788 thread_start + 8com.apple.avplayeritem.figplaybackitem.accessor

com.apple.avplayeritem.figplaybackitem.accessor
0  libsystem_kernel.dylib         0x1b26137ac __psynch_mutexwait + 8
1  libsystem_pthread.dylib        0x1cdc402f4 _pthread_mutex_firstfit_lock_wait + 92
2  libsystem_pthread.dylib        0x1cdc40244 _pthread_mutex_firstfit_lock_slow + 216
3  MediaToolbox                   0x18eb8c710 playerasync_runImmediateCommand + 172
4  MediaToolbox                   0x18ea84794 itemasync_CopyProperty + 524
5  AVFCore                        0x18df56dbc +[AVPlayerItem _copyTimebaseFromFigPlaybackItem:] + 176
6  AVFCore                        0x18df57154 __31-[AVPlayerItem _updateTimebase]_block_invoke + 80
7  libdispatch.dylib              0x1862defd0 _dispatch_call_block_and_release + 32
8  libdispatch.dylib              0x1862e0ac8 _dispatch_client_callout + 20
9  libdispatch.dylib              0x1862e7c08 _dispatch_lane_serial_drain + 580
10 libdispatch.dylib              0x1862e8734 _dispatch_lane_invoke + 408
11 libdispatch.dylib              0x1862f2528 _dispatch_workloop_worker_thread + 708
12 libsystem_pthread.dylib        0x1cdc3c908 _pthread_wqthread + 276
13 libsystem_pthread.dylib        0x1cdc4377c start_wqthread + 8Thread #4

Thread
0  libsystem_kernel.dylib         0x1b25efe24 semaphore_timedwait_trap + 8
1  libdispatch.dylib              0x1862e1098 _dispatch_sema4_timedwait + 68
2  libdispatch.dylib              0x1862e1678 _dispatch_semaphore_wait_slow + 76
3  libdispatch.dylib              0x1862f0f00 _dispatch_worker_thread + 308
4  libsystem_pthread.dylib        0x1cdc3aca8 _pthread_start + 320
5  libsystem_pthread.dylib        0x1cdc43788 thread_start + 8com.apple.coremedia.player.async.0x2831b4840.P/YR

com.apple.coremedia.player.async.0x2831b4840.P/YR
0  libsystem_kernel.dylib         0x1b26143d0 __ulock_wait + 8
1  libsystem_platform.dylib       0x1cdc350e8 _os_unfair_lock_lock_slow + 196
2  libobjc.A.dylib                0x19a77b0bc objc_object::sidetable_clearDeallocating() + 100
3  libobjc.A.dylib                0x19a771d74 objc_destructInstance + 140
4  libobjc.A.dylib                0x19a77899c _objc_rootDealloc + 68
5  libxpc.dylib                   0x1cdc789c0 -[OS_xpc_object dealloc] + 56
6  libxpc.dylib                   0x1cdc6754c _xpc_dictionary_node_free + 84
7  libxpc.dylib                   0x1cdc64670 _xpc_dictionary_dispose + 48
8  libxpc.dylib                   0x1cdc789a4 -[OS_xpc_object dealloc] + 28
9  libxpc.dylib                   0x1cdc6754c _xpc_dictionary_node_free + 84
10 libxpc.dylib                   0x1cdc64670 _xpc_dictionary_dispose + 48
11 libxpc.dylib                   0x1cdc789a4 -[OS_xpc_object dealloc] + 28
12 CoreMedia                      0x18f2ee3d8 FigXPCSendStdSetPropertyMessage + 144
13 MediaToolbox                   0x18ec8bc88 remoteXPCPlayer_SetProperty + 152
14 MediaToolbox                   0x18eb8dae8 playerasync_runOneCommand + 4836
15 MediaToolbox                   0x18eb908b8 playerasync_runAsynchronousCommandOnQueue + 176
16 libdispatch.dylib              0x1862e0ac8 _dispatch_client_callout + 20
17 libdispatch.dylib              0x1862e7c08 _dispatch_lane_serial_drain + 580
18 libdispatch.dylib              0x1862e8768 _dispatch_lane_invoke + 460
19 libdispatch.dylib              0x1862f1104 _dispatch_root_queue_drain + 356
20 libdispatch.dylib              0x1862f0ebc _dispatch_worker_thread + 240
21 libsystem_pthread.dylib        0x1cdc3aca8 _pthread_start + 320
22 libsystem_pthread.dylib        0x1cdc43788 thread_start + 8

[nuno-vieira]

In Xcode the crash is a bit cleaner:

I'll try and get the full crash report from the dev that encountered the crash.

Update:

Thread 1 Queue : com.apple.main-thread (serial)
#0  0x00000001b7129184 in _os_unfair_lock_recursive_abort ()
#1  0x00000001b712706c in _os_unfair_lock_lock_slow ()
#2  0x00000001b7156c1c in objc_object::sidetable_clearDeallocating() ()
#3  0x00000001b714e058 in objc_destructInstance ()
#4  0x00000001c4f074bc in swift_deallocClassInstance ()
#5  0x00000001c4f073e0 in _swift_release_dealloc ()
#6  0x0000000103e91a7c in (anonymous namespace)::freeContainer(ContainerHeader*) ()
#7  0x0000000103e902f8 in (anonymous namespace)::garbageCollect(MemoryState*, bool) ()
#8  0x0000000103e954e0 in ReleaseHeapRefStrict ()
#9  0x0000000103e92fb8 in BackRefFromAssociatedObject::tryAddRef() ()
#10 0x0000000103e8d088 in (anonymous namespace)::_tryRetainImp(objc_object*, objc_selector*) ()
#11 0x00000001b7156820 in objc_loadWeakRetained ()
#12 0x0000000103f0362c in -[ADBMediaHeartbeat qosInfo] ()
#13 0x0000000103ed1900 in -[ADB_VHB_MediaHeartbeatVideoPlayerPluginDelegate getQoSInfo] ()
#14 0x0000000103f191b4 in -[ADB_VHB_VideoPlayerPlugin cacheQoSInfo] ()
#15 0x0000000103f184ec in __46-[ADB_VHB_VideoPlayerPlugin setupDataResolver]_block_invoke549 ()
#16 0x0000000103ef1588 in -[ADB_HB_Callback call:] ()
#17 0x0000000103f188f8 in __46-[ADB_VHB_VideoPlayerPlugin setupDataResolver]_block_invoke567 ()
#18 0x0000000103ef1588 in -[ADB_HB_Callback call:] ()
#19 0x0000000103ef0d44 in -[ADB_HB_BasePlugin resolveData:] ()
#20 0x0000000103f1af70 in -[ADB_HB_PluginManager request:keys:] ()
#21 0x0000000103f1a9d4 in -[ADB_HB_PluginManager triggerEvent:] ()
#22 0x0000000103ef128c in -[ADB_HB_BasePlugin _trigger:info:] ()
#23 0x0000000103f13e00 in -[ADB_VHB_VideoPlayerPlugin trackAdBreakComplete] ()
#24 0x0000000103ef9080 in -[ADBMediaHeartbeat _cmdAdBreakComplete:] ()
#25 0x0000000103f02340 in __32-[ADBMediaHeartbeat _setupRules]_block_invoke777 ()
#26 0x0000000103ef1588 in -[ADB_HB_Callback call:] ()
#27 0x0000000103ee9074 in -[ADBRuleEngine processRule:ruleEngineContext:] ()
#28 0x0000000103efa59c in -[ADBMediaHeartbeat processRule:ruleEngineContext:] ()
#29 0x0000000103ef72b8 in -[ADBMediaHeartbeat trackEvent:mediaObject:data:] ()
#30 0x0000000103c47744 in kfun:com.sky.core.player.sdk.addon.adobe.AdobeMediaHeartbeatWrapperImpl.trackEvent#internal at /Users/cwn16/Developer/Projects/core-video-sdk-multiplatform/AddonManager/src/appleMain/kotlin/com/sky/core/player/sdk/addon/adobe/AdobeMediaHeartbeatWrapperImpl.kt:204
#31 0x0000000103c464ac in kfun:com.sky.core.player.sdk.addon.adobe.AdobeMediaHeartbeatWrapperImpl#trackAdvertBreakEnded(com.sky.core.player.sdk.addon.data.AdBreakData){} at /Users/cwn16/Developer/Projects/core-video-sdk-multiplatform/AddonManager/src/appleMain/kotlin/com/sky/core/player/sdk/addon/adobe/AdobeMediaHeartbeatWrapperImpl.kt:133
#32 0x0000000103c835ac in <anonymous> [inlined] at /Users/cwn16/Developer/Projects/core-video-sdk-multiplatform/AddonManager/src/commonMain/kotlin/com/sky/core/player/sdk/addon/adobe/AdobeMediaAddon.kt:221
#33 0x0000000103c8357c in let [inlined] at /Users/teamcity/buildAgent/work/cae0e6559deed4c4/backend.native/build/stdlib/kotlin/util/Standard.kt:112
#34 0x0000000103c8357c in kfun:com.sky.core.player.sdk.addon.adobe.AdobeMediaAddon#onAdBreakEnded(com.sky.core.player.sdk.addon.data.AdBreakData){} at /Users/cwn16/Developer/Projects/core-video-sdk-multiplatform/AddonManager/src/commonMain/kotlin/com/sky/core/player/sdk/addon/adobe/AdobeMediaAddon.kt:220
#35 0x0000000103c76990 in onAdBreakEnded$lambda-12 [inlined] at /Users/cwn16/Developer/Projects/core-video-sdk-multiplatform/AddonManager/src/commonMain/kotlin/com/sky/core/player/sdk/addon/AddonManagerImpl.kt:242
#36 0x0000000103c76964 in invoke [inlined] at /Users/cwn16/Developer/Projects/core-video-sdk-multiplatform/AddonManager/src/commonMain/kotlin/com/sky/core/player/sdk/addon/AddonManagerImpl.kt:242
#37 0x0000000103c76954 in kfun:com.sky.core.player.sdk.addon.AddonManagerImpl.$onAdBreakEnded$lambda-12$FUNCTION_REFERENCE$24.$<bridge-UNNN>invoke(-1:0){}#internal at /Users/cwn16/Developer/Projects/core-video-sdk-multiplatform/AddonManager/src/commonMain/kotlin/com/sky/core/player/sdk/addon/AddonManagerImpl.kt:242
#38 0x0000000103c72434 in forEach [inlined] at /Users/teamcity/buildAgent/work/cae0e6559deed4c4/backend.native/build/stdlib/generated/_Collections.kt:1799
#39 0x0000000103c7237c in kfun:com.sky.core.player.sdk.addon.AddonManagerImpl.forEachAdListenerAddon#internal at /Users/cwn16/Developer/Projects/core-video-sdk-multiplatform/AddonManager/src/commonMain/kotlin/com/sky/core/player/sdk/addon/AddonManagerImpl.kt:457
#40 0x0000000103c6e184 in kfun:com.sky.core.player.sdk.addon.AddonManagerImpl#onAdBreakEnded(com.sky.core.player.sdk.addon.data.AdBreakData){} at /Users/cwn16/Developer/Projects/core-video-sdk-multiplatform/AddonManager/src/commonMain/kotlin/com/sky/core/player/sdk/addon/AddonManagerImpl.kt:242
#41 0x0000000103db24f4 in objc2kotlin.1424 at /<compiler-generated>:1
#42 0x000000010333871c in closure #1 in AddonManagerFacade.onAdBreakFinished(adBreak:) at /Users/RGA06/workspace/core-video-sdk-ios/CoreVideoSDK/CoreVideoSDK/AddonManager/AddonManagerFacade.swift:365
#43 0x0000000103075ce4 in thunk for @escaping @callee_guaranteed () -> () ()
#44 0x000000010722605c in _dispatch_call_block_and_release ()
#45 0x00000001072274d8 in _dispatch_client_callout ()
#46 0x0000000107235f64 in _dispatch_main_queue_callback_4CF ()
#47 0x00000001b73a08d4 in __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__ ()
#48 0x00000001b739b58c in __CFRunLoopRun ()
#49 0x00000001b739abc8 in CFRunLoopRunSpecific ()
#50 0x00000001c17835cc in GSEventRunModal ()
#51 0x00000001bb54d744 in UIApplicationMain ()
#52 0x0000000102932664 in main at /Users/RGA06/workspace/core-video-sdk-ios/ReferenceApps/ReferenceAppiOS/ReferenceAppShared/AppDelegate.swift:8
#53 0x00000001b7217384 in start ()
Enqueued from com.apple.main-thread (Thread 1) Queue : com.apple.main-thread (serial)
#0  0x000000010722bdd0 in dispatch_async ()
#1  0x00000001ee24c04c in OS_dispatch_queue.async(group:qos:flags:execute:) ()
#2  0x00000001033385f0 in AddonManagerFacade.onAdBreakFinished(adBreak:) at /Users/RGA06/workspace/core-video-sdk-ios/CoreVideoSDK/CoreVideoSDK/AddonManager/AddonManagerFacade.swift:364
#3  0x0000000103154c94 in SessionController.onAdBreakFinished(adBreak:) at /Users/RGA06/workspace/core-video-sdk-ios/CoreVideoSDK/CoreVideoSDK/SessionController/Internal/SessionController+AdvertListener.swift:58
#4  0x0000000103155664 in protocol witness for AdvertListener.onAdBreakFinished(adBreak:) in conformance SessionController ()
#5  0x000000010327b2ec in HelioPlayerEngine.didReceiveNewAdBreaks(_:) at /Users/RGA06/workspace/core-video-sdk-ios/CoreVideoSDK/CoreVideoSDK_Helio/PlayerEngine/HelioPlayerEngine+LiveAdDecisioningHandlerDelegate.swift:25
#6  0x000000010327add4 in closure #1 in closure #1 in HelioPlayerEngine.didReceiveLiveAdvertsRequestParameters(_:completion:) at /Users/RGA06/workspace/core-video-sdk-ios/CoreVideoSDK/CoreVideoSDK_Helio/PlayerEngine/HelioPlayerEngine+LiveAdDecisioningHandlerDelegate.swift:11
#7  0x00000001030dffc8 in closure #1 in SessionController.didReceiveLiveAdvertsRequestParameters(_:completion:) at /Users/RGA06/workspace/core-video-sdk-ios/CoreVideoSDK/CoreVideoSDK/SessionController/Internal/SessionController+PlayerEngineListener.swift:29
#8  0x0000000103331dec in closure #1 in closure #1 in AddonManagerFacade.getCSAIAdverts(withVacResponse:completion:) at /Users/RGA06/workspace/core-video-sdk-ios/CoreVideoSDK/CoreVideoSDK/AddonManager/AddonManagerFacade.swift:145
#9  0x0000000103331798 in thunk for @escaping @callee_guaranteed (@guaranteed Result<AddonManagerAdBreakDataHolder, KotlinError<AddonManagerAdBreakDataHolder>>) -> () ()
#10 0x0000000103314974 in closure #1 in AddonManagerKotlinx_coroutines_coreDeferred.await<A>(_:_:) at /Users/RGA06/workspace/core-video-sdk-ios/CoreVideoSDK/CoreVideoSDK/KotlinMappers/KotlinExtensions.swift:27
#11 0x0000000103314bd0 in thunk for @escaping @callee_guaranteed (@guaranteed AddonManagerKotlinThrowable?) -> () ()
#12 0x0000000103daaea8 in invokeFunction1V ()
#13 0x0000000103b479cc in kfun:kotlinx.coroutines.JobSupport.completeStateFinalization#internal at /opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:321
#14 0x0000000103b4d32c in tryFinalizeSimpleState [inlined] at /opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:297
#15 0x0000000103b4d2ac in kfun:kotlinx.coroutines.JobSupport.tryMakeCompleting#internal at /opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:868
#16 0x0000000103b35e98 in <anonymous> [inlined] at /opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:840
#17 0x0000000103b35e8c in loopOnState [inlined] at /opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:178
#18 0x0000000103b35e8c in makeCompletingOnce [inlined] at /opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:839
#19 0x0000000103b35e00 in kfun:kotlinx.coroutines.AbstractCoroutine#resumeWith(kotlin.Result<1:0>){} at /opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/AbstractCoroutine.kt:111
#20 0x0000000103a4b7c4 in <anonymous> [inlined] at /Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/coroutines/ContinuationImpl.kt:43
#21 0x0000000103a4b6f8 in with [inlined] at /Users/teamcity/buildAgent/work/cae0e6559deed4c4/backend.native/build/stdlib/kotlin/util/Standard.kt:70
#22 0x0000000103a4b6f8 in kfun:kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(kotlin.Result<kotlin.Any?>){} at /Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/coroutines/ContinuationImpl.kt:26
#23 0x0000000103b6ac1c in resume [inlined] at /Users/teamcity/buildAgent/work/cae0e6559deed4c4/backend.native/build/stdlib/kotlin/coroutines/Continuation.kt:45
#24 0x0000000103b6abe8 in <anonymous> [inlined] at /opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/internal/DispatchedTask.kt:56
#25 0x0000000103b6abcc in withCoroutineContext [inlined] at /opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/CoroutineContext.kt:37
#26 0x0000000103b6abcc in kfun:kotlinx.coroutines.DispatchedTask#run(){} at /opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/internal/DispatchedTask.kt:42
#27 0x0000000103c420e8 in dispatch$lambda-0 [inlined] at /Users/cwn16/Developer/Projects/core-video-sdk-multiplatform/AddonManager/src/appleMain/kotlin/com/sky/core/player/sdk/addon/GetCoroutineDispatcher.kt:73
#28 0x0000000103c420b8 in invoke [inlined] at /Users/cwn16/Developer/Projects/core-video-sdk-multiplatform/AddonManager/src/appleMain/kotlin/com/sky/core/player/sdk/addon/GetCoroutineDispatcher.kt:72
#29 0x0000000103c420b0 in kfun:com.sky.core.player.sdk.addon.MainLoopDispatcher.$dispatch$lambda-0$FUNCTION_REFERENCE$0.$<bridge-UNN>invoke(){}#internal at /Users/cwn16/Developer/Projects/core-video-sdk-multiplatform/AddonManager/src/appleMain/kotlin/com/sky/core/player/sdk/addon/GetCoroutineDispatcher.kt:72
#30 0x0000000103e83a3c in ___636f72652d766964656f2d73646b2d6d756c7469706c6174666f726d3a4164646f6e4d616e61676572_knbridge6_block_invoke ()
#31 0x000000010722605c in _dispatch_call_block_and_release ()
#32 0x00000001072274d8 in _dispatch_client_callout ()
#33 0x0000000107235f64 in _dispatch_main_queue_callback_4CF ()
#34 0x00000001b73a08d4 in __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__ ()
#35 0x00000001b739b58c in __CFRunLoopRun ()
#36 0x00000001b739abc8 in CFRunLoopRunSpecific ()
#37 0x00000001c17835cc in GSEventRunModal ()
#38 0x00000001bb54d744 in UIApplicationMain ()
#39 0x0000000102932664 in main at /Users/RGA06/workspace/core-video-sdk-ios/ReferenceApps/ReferenceAppiOS/ReferenceAppShared/AppDelegate.swift:8
#40 0x00000001b7217384 in start ()
Enqueued from com.apple.main-thread (Thread 1) Queue : com.apple.main-thread (serial)
#0  0x000000010722bdd0 in dispatch_async ()
#1  0x0000000103c41a40 in kfun:com.sky.core.player.sdk.addon.MainLoopDispatcher.dispatch#internal at /Users/cwn16/Developer/Projects/core-video-sdk-multiplatform/AddonManager/src/appleMain/kotlin/com/sky/core/player/sdk/addon/GetCoroutineDispatcher.kt:72
#2  0x0000000103b69a2c in kfun:kotlinx.coroutines.DispatchedContinuation#resumeWith(kotlin.Result<1:0>){} at /opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/internal/DispatchedContinuation.kt:184
#3  0x0000000103a4c2f0 in kfun:kotlin.coroutines.SafeContinuation#resumeWith(kotlin.Result<1:0>){} at /Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/coroutines/SafeContinuationNative.kt:33
#4  0x0000000103c56100 in resume [inlined] at /Users/teamcity/buildAgent/work/cae0e6559deed4c4/backend.native/build/stdlib/kotlin/coroutines/Continuation.kt:45
#5  0x0000000103c560cc in kfun:com.sky.core.player.sdk.addon.networkLayer.NativeNetworkApi.object-1.URLSession#internal.310 at /Users/cwn16/Developer/Projects/core-video-sdk-multiplatform/AddonManager/src/appleMain/kotlin/com/sky/core/player/sdk/addon/networkLayer/NativeNetworkApi.kt:76
#6  0x0000000103e8467c in _636f72652d766964656f2d73646b2d6d756c7469706c6174666f726d3a4164646f6e4d616e61676572_kncfun202 ()
#7  0x00000001ba856904 in ___lldb_unnamed_symbol9163$$CFNetwork ()
#8  0x00000001b7806004 in __NSBLOCKOPERATION_IS_CALLING_OUT_TO_A_BLOCK__ ()
#9  0x00000001b77085a4 in -[NSBlockOperation main] ()
#10 0x00000001b7808344 in __NSOPERATION_IS_INVOKING_MAIN__ ()
#11 0x00000001b770827c in -[NSOperation start] ()
#12 0x00000001b7808d70 in __NSOPERATIONQUEUE_IS_STARTING_AN_OPERATION__ ()
#13 0x00000001b7808830 in __NSOQSchedule_f ()
#14 0x00000001072367c4 in _dispatch_block_async_invoke2 ()
#15 0x00000001072274d8 in _dispatch_client_callout ()
#16 0x0000000107235f64 in _dispatch_main_queue_callback_4CF ()
#17 0x00000001b73a08d4 in __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__ ()
#18 0x00000001b739b58c in __CFRunLoopRun ()
#19 0x00000001b739abc8 in CFRunLoopRunSpecific ()
#20 0x00000001c17835cc in GSEventRunModal ()
#21 0x00000001bb54d744 in UIApplicationMain ()
#22 0x0000000102932664 in main at /Users/RGA06/workspace/core-video-sdk-ios/ReferenceApps/ReferenceAppiOS/ReferenceAppShared/AppDelegate.swift:8
#23 0x00000001b7217384 in start ()
Enqueued from com.apple.NSURLSession-work (Thread 21) Queue : com.apple.NSURLSession-work (serial)
#0  0x000000010722bdd0 in dispatch_async ()
#1  0x00000001b78050d4 in __NSOQSchedule ()
#2  0x00000001b7806b40 in __addOperations ()
#3  0x00000001ba6a6f88 in ___lldb_unnamed_symbol94$$CFNetwork ()
#4  0x00000001ba6a4234 in ___lldb_unnamed_symbol68$$CFNetwork ()
#5  0x00000001ba6a3588 in ___lldb_unnamed_symbol64$$CFNetwork ()
#6  0x00000001ba742ad4 in ___lldb_unnamed_symbol3144$$CFNetwork ()
#7  0x00000001ba6a2f54 in ___lldb_unnamed_symbol59$$CFNetwork ()
#8  0x00000001ba6a4684 in ___lldb_unnamed_symbol74$$CFNetwork ()
#9  0x00000001ba7440fc in ___lldb_unnamed_symbol3221$$CFNetwork ()
#10 0x00000001072367c4 in _dispatch_block_async_invoke2 ()
#11 0x00000001072274d8 in _dispatch_client_callout ()
#12 0x000000010722ec20 in _dispatch_lane_serial_drain ()
#13 0x000000010722f868 in _dispatch_lane_invoke ()
#14 0x000000010723b270 in _dispatch_workloop_worker_thread ()
#15 0x00000001b712d718 in _pthread_wqthread ()
#16 0x00000001b71339c8 in start_wqthread ()
Enqueued from com.apple.NSURLSession-work (Thread 21) Queue : com.apple.NSURLSession-work (serial)
#0  0x000000010722bdd0 in dispatch_async ()
#1  0x00000001ba6a042c in ___lldb_unnamed_symbol21$$CFNetwork ()
#2  0x00000001ba7440c4 in ___lldb_unnamed_symbol3220$$CFNetwork ()
#3  0x00000001ba810530 in ___lldb_unnamed_symbol7441$$CFNetwork ()
#4  0x00000001ba807b80 in ___lldb_unnamed_symbol7216$$CFNetwork ()
#5  0x000000010722605c in _dispatch_call_block_and_release ()
#6  0x00000001072274d8 in _dispatch_client_callout ()
#7  0x000000010722ec20 in _dispatch_lane_serial_drain ()
#8  0x000000010722f868 in _dispatch_lane_invoke ()
#9  0x000000010723b270 in _dispatch_workloop_worker_thread ()
#10 0x00000001b712d718 in _pthread_wqthread ()
#11 0x00000001b71339c8 in start_wqthread ()
Enqueued from com.apple.CFNetwork.Connection (Thread 7) Queue : com.apple.CFNetwork.Connection (serial)
#0  0x000000010722bdd0 in dispatch_async ()
#1  0x00000001ba807b48 in ___lldb_unnamed_symbol7215$$CFNetwork ()
#2  0x00000001ba815cd4 in ___lldb_unnamed_symbol7646$$CFNetwork ()
#3  0x00000001ba70e024 in ___lldb_unnamed_symbol2146$$CFNetwork ()
#4  0x00000001ba7c9d64 in ___lldb_unnamed_symbol5825$$CFNetwork ()
#5  0x00000001ba8aafbc in ___lldb_unnamed_symbol11379$$CFNetwork ()
#6  0x00000001ba846ce8 in ___lldb_unnamed_symbol8591$$CFNetwork ()
#7  0x00000001ba8a28cc in ___lldb_unnamed_symbol11128$$CFNetwork ()
#8  0x00000001ba8a3658 in ___lldb_unnamed_symbol11147$$CFNetwork ()
#9  0x00000001ba8a4d3c in ___lldb_unnamed_symbol11168$$CFNetwork ()
#10 0x00000001ba8a49cc in ___lldb_unnamed_symbol11162$$CFNetwork ()
#11 0x00000001ba8a38ec in ___lldb_unnamed_symbol11149$$CFNetwork ()
#12 0x00000001ba8a3fac in ___lldb_unnamed_symbol11150$$CFNetwork ()
#13 0x00000001ba8a36f4 in ___lldb_unnamed_symbol11147$$CFNetwork ()
#14 0x00000001ba8a3138 in ___lldb_unnamed_symbol11134$$CFNetwork ()
#15 0x000000010722605c in _dispatch_call_block_and_release ()
#16 0x00000001072274d8 in _dispatch_client_callout ()
#17 0x000000010722ec20 in _dispatch_lane_serial_drain ()
#18 0x000000010722f868 in _dispatch_lane_invoke ()
#19 0x0000000107230c3c in _dispatch_workloop_invoke ()
#20 0x000000010723b270 in _dispatch_workloop_worker_thread ()
#21 0x00000001b712d718 in _pthread_wqthread ()
#22 0x00000001b71339c8 in start_wqthread ()
Enqueued from com.apple.CFNetwork.Connection (Thread 7) Queue : com.apple.CFNetwork.Connection (serial)
#0  0x000000010722bdd0 in dispatch_async ()
#1  0x00000001ba8a2f58 in ___lldb_unnamed_symbol11131$$CFNetwork ()
#2  0x00000001ba8a3b80 in ___lldb_unnamed_symbol11150$$CFNetwork ()
#3  0x00000001ba8a36f4 in ___lldb_unnamed_symbol11147$$CFNetwork ()
#4  0x00000001ba8a3138 in ___lldb_unnamed_symbol11134$$CFNetwork ()
#5  0x000000010722605c in _dispatch_call_block_and_release ()
#6  0x00000001072274d8 in _dispatch_client_callout ()
#7  0x000000010722ec20 in _dispatch_lane_serial_drain ()
#8  0x000000010722f868 in _dispatch_lane_invoke ()
#9  0x0000000107230c3c in _dispatch_workloop_invoke ()
#10 0x000000010723b270 in _dispatch_workloop_worker_thread ()
#11 0x00000001b712d718 in _pthread_wqthread ()
#12 0x00000001b71339c8 in start_wqthread ()
Enqueued from com.apple.CFNetwork.Connection (Thread 7) Queue : com.apple.CFNetwork.Connection (serial)
#0  0x000000010722bdd0 in dispatch_async ()
#1  0x00000001ba8a2f58 in ___lldb_unnamed_symbol11131$$CFNetwork ()
#2  0x00000001ba8a3b80 in ___lldb_unnamed_symbol11150$$CFNetwork ()
#3  0x00000001ba8a07bc in ___lldb_unnamed_symbol11087$$CFNetwork ()
#4  0x00000001ba8a002c in ___lldb_unnamed_symbol11073$$CFNetwork ()
#5  0x00000001ba851d40 in ___lldb_unnamed_symbol8997$$CFNetwork ()
#6  0x000000010722605c in _dispatch_call_block_and_release ()
#7  0x00000001072274d8 in _dispatch_client_callout ()
#8  0x000000010722ec20 in _dispatch_lane_serial_drain ()
#9  0x000000010722f868 in _dispatch_lane_invoke ()
#10 0x0000000107230c3c in _dispatch_workloop_invoke ()
#11 0x000000010723b270 in _dispatch_workloop_worker_thread ()
#12 0x00000001b712d718 in _pthread_wqthread ()
#13 0x00000001b71339c8 in start_wqthread ()
Enqueued from com.apple.CFNetwork.Connection (Thread 7) Queue : com.apple.CFNetwork.Connection (serial)
#0  0x000000010722bdd0 in dispatch_async ()
#1  0x00000001ba85204c in ___lldb_unnamed_symbol8999$$CFNetwork ()
#2  0x00000001ba74ebf4 in ___lldb_unnamed_symbol3383$$CFNetwork ()
#3  0x00000001072367c4 in _dispatch_block_async_invoke2 ()
#4  0x00000001072274d8 in _dispatch_client_callout ()
#5  0x000000010722ec20 in _dispatch_lane_serial_drain ()
#6  0x000000010722f868 in _dispatch_lane_invoke ()
#7  0x0000000107230c3c in _dispatch_workloop_invoke ()
#8  0x000000010723b270 in _dispatch_workloop_worker_thread ()
#9  0x00000001b712d718 in _pthread_wqthread ()
#10 0x00000001b71339c8 in start_wqthread ()
Enqueued from com.apple.network.connections (Thread 7) Queue : com.apple.network.connections (serial)
#0  0x000000010722bdd0 in dispatch_async ()
#1  0x00000001b97e03b8 in nw_read_request_report ()
#2  0x00000001b97e1710 in nw_read_request_list_report ()
#3  0x00000001b993730c in nw_flow_process_input_frames ()
#4  0x00000001b9933d10 in nw_flow_service_reads ()
#5  0x00000001b9948a88 in nw_flow_input_available ()
#6  0x00000001bc3b92b8 in nw_boringssl_read ()
#7  0x00000001bc3ba0dc in nw_protocol_boringssl_input_available ()
#8  0x00000001bc816114 in nw_protocol_tcp_wake_read ()
#9  0x00000001bc81c360 in nw_protocol_tcp_input_flush ()
#10 0x00000001b9875c04 in nw_channel_add_input_frames ()
#11 0x00000001b98749d0 in nw_channel_update_input_source ()
#12 0x00000001b98741e8 in __nw_channel_create_block_invoke.32 ()
#13 0x00000001072274d8 in _dispatch_client_callout ()
#14 0x000000010722a320 in _dispatch_continuation_pop ()
#15 0x000000010723e224 in _dispatch_source_invoke ()
#16 0x0000000107230c3c in _dispatch_workloop_invoke ()
#17 0x000000010723b270 in _dispatch_workloop_worker_thread ()
#18 0x00000001b712d718 in _pthread_wqthread ()
#19 0x00000001b71339c8 in start_wqthread ()
Thread 2Thread 3Thread 4Thread 5 Queue : com.sky.core.player.sdk.helio.live-ad-decisioning-handler (serial)
#0  0x00000001b720d7e4 in __ulock_wait ()
#1  0x0000000107227ff8 in _dlock_wait ()
#2  0x0000000107228390 in _dispatch_group_wait_slow ()
#3  0x000000010310be64 in closure #4 in EventAdDecisioningHandler.handle(decisionData:withMetadata:withAdOpportunityData:) at /Users/RGA06/workspace/core-video-sdk-ios/CoreVideoSDK/CoreVideoSDK_Helio/Adverts/EventAdDecisioningHandler.swift:116
#4  0x0000000103075ce4 in thunk for @escaping @callee_guaranteed () -> () ()
#5  0x000000010722605c in _dispatch_call_block_and_release ()
#6  0x00000001072274d8 in _dispatch_client_callout ()
#7  0x000000010722ec20 in _dispatch_lane_serial_drain ()
#8  0x000000010722f834 in _dispatch_lane_invoke ()
#9  0x000000010723b270 in _dispatch_workloop_worker_thread ()
#10 0x00000001b712d718 in _pthread_wqthread ()
Enqueued from com.comcast.helio.AdDecisioningManager.adHandlerRequest.https://g002-sle-us-cmaf-prd-ak.cdn.peacocktv.com/live/slemorningnewsf4 (Thread 5) Queue : com.comcast.helio.AdDecisioningManager.adHandlerRequest.https://g002-sle-us-cmaf-prd-ak.cdn.peacocktv.com/live/slemorningnewsf4 (serial)
#0  0x000000010722bdd0 in dispatch_async ()
#1  0x00000001ee24c04c in OS_dispatch_queue.async(group:qos:flags:execute:) ()
#2  0x000000010310a6c8 in EventAdDecisioningHandler.handle(decisionData:withMetadata:withAdOpportunityData:) at /Users/RGA06/workspace/core-video-sdk-ios/CoreVideoSDK/CoreVideoSDK_Helio/Adverts/EventAdDecisioningHandler.swift:71
#3  0x000000010310fb7c in protocol witness for AdDecisioningHandler.handle(decisionData:withMetadata:withAdOpportunityData:) in conformance EventAdDecisioningHandler ()
#4  0x00000001073f641c in _AnyAdDecisioningHandlerBox.handle(decisionData:withMetadata:withAdOpportunityData:) at /Users/RGA06/workspace/core-video-sdk-ios/Carthage/Checkouts/helio_ios/helioSharedFramework/Ad Insertion/Protocol Type Erasers/AnyAdDecisioningHandler.swift:149
#5  0x00000001073f58d0 in AnyAdDecisioningHandler.handle(decisionData:withMetadata:withAdOpportunityData:) at /Users/RGA06/workspace/core-video-sdk-ios/Carthage/Checkouts/helio_ios/helioSharedFramework/Ad Insertion/Protocol Type Erasers/AnyAdDecisioningHandler.swift:49
#6  0x00000001072f86c0 in closure #1 in closure #1 in closure #1 in AdDecisioningManager.observePlaylistTransform(atUrl:forPlaylist:withObserver:withMetadata:) at /Users/RGA06/workspace/core-video-sdk-ios/Carthage/Checkouts/helio_ios/helioSharedFramework/Ad Insertion/AdDecisioningManager.swift:170
#7  0x000000010730fb18 in thunk for @escaping @callee_guaranteed () -> () ()
#8  0x000000010722605c in _dispatch_call_block_and_release ()
#9  0x00000001072274d8 in _dispatch_client_callout ()
#10 0x000000010722ec20 in _dispatch_lane_serial_drain ()
#11 0x000000010722f834 in _dispatch_lane_invoke ()
#12 0x000000010723b270 in _dispatch_workloop_worker_thread ()
#13 0x00000001b712d718 in _pthread_wqthread ()
#14 0x00000001b71339c8 in start_wqthread ()
Enqueued from com.comcast.helio.AdDecisioningManager.availability.https://g002-sle-us-cmaf-prd-ak.cdn.peacocktv.com/live/slemorningnewsf4f961 (Thread 5) Queue : com.comcast.helio.AdDecisioningManager.availability.https://g002-sle-us-cmaf-prd-ak.cdn.peacocktv.com/live/slemorningnewsf4f961 (serial)
#0  0x000000010722bdd0 in dispatch_async ()
#1  0x00000001ee24c04c in OS_dispatch_queue.async(group:qos:flags:execute:) ()
#2  0x00000001072f7cfc in closure #1 in closure #1 in AdDecisioningManager.observePlaylistTransform(atUrl:forPlaylist:withObserver:withMetadata:) at /Users/RGA06/workspace/core-video-sdk-ios/Carthage/Checkouts/helio_ios/helioSharedFramework/Ad Insertion/AdDecisioningManager.swift:149
#3  0x000000010730fb18 in thunk for @escaping @callee_guaranteed () -> () ()
#4  0x000000010722605c in _dispatch_call_block_and_release ()
#5  0x00000001072274d8 in _dispatch_client_callout ()
#6  0x000000010722a320 in _dispatch_continuation_pop ()
#7  0x0000000107229704 in _dispatch_async_redirect_invoke ()
#8  0x0000000107239850 in _dispatch_root_queue_drain ()
#9  0x000000010723a1b8 in _dispatch_worker_thread2 ()
#10 0x00000001b712d6dc in _pthread_wqthread ()
#11 0x00000001b71339c8 in start_wqthread ()
com.apple.uikit.eventfetch-thread (6)Thread 7 Queue : Anonymous
Thread 8com.apple.NSURLConnectionLoader (10)AVAudioSession Notify Thread (11)Thread 12JavaScriptCore bmalloc scavenger (13)ComScore-Thread (14)ComScore-NetworkThread (15)ComScore-StreamingAnalytics-1 (16)Thread 18 Queue : com.conviva.ConvivaUtils.cisSharedQueue (serial)
Thread 19Thread 20 Queue : com.apple.NSURLSession-work (serial)
Thread 21Thread 22Thread 23Thread 24

[praveek]

@nuno-vieira Thanks for the detailed report. I will investigate the crash and update this issue. To answer your question, cacheQoSInfo and cachePlayhead methods calls the getCurrentPlaybackTime and getQoSObject methods of the delegate passed when creating MediaHeartbeat instance.

[nuno-vieira]

@nuno-vieira Thanks for the detailed report. I will investigate the crash and update this issue. To answer your question, cacheQoSInfo and cachePlayhead methods calls the getCurrentPlaybackTime and getQoSObject methods of the delegate passed when creating MediaHeartbeat instance.

@praveek If there is missing data from the QoSObject, what happens? Is there any exception or assertion thrown?

Also, I have some logs from Adobe, that might be useful for you guys.

In these logs we play an asset, then an advert break starts, and after it ends, the session is closed.

Something that concerns me about these logs, is this:

2020-11-02 20:09:17.331488+0000 ReferenceAppiOS_Helio[5423:5965472] [INFO] [2020-11-02 at 20:09:17][ADB_VHB_Heartbeat] #destroy()
2020-11-02 20:09:17.343273+0000 ReferenceAppiOS_Helio[5423:5965472] [INFO] [2020-11-02 at 20:09:17][ADB_VHB_Heartbeat] #dealloc()
2020-11-02 20:09:17.344116+0000 ReferenceAppiOS_Helio[5423:5965472] [WARN] [2020-11-02 at 20:09:17][ADB_VHB_Heartbeat] #destroy() > Instance already destroyed.

It looks that the Heartbeat instance is being destroyed twice, where it was already destroyed. How could this happen? We do not deref the Heartbeat instance manually. In a playback session, the object that holds the heartbeat is destroyed. And there's only one object holding the heartbeat instance.

Full Adobe Logs: AdobeFullLog.txt

[praveek]

We recommend setting missing data to zero when creating QoSObject and it is handled by the SDK.

2020-11-02 20:09:17.344116+0000 ReferenceAppiOS_Helio[5423:5965472] [WARN] [2020-11-02 at 20:09:17][ADB_VHB_Heartbeat] #destroy() > Instance already destroyed. This log can be safely ignored. The destroy method frees some resources and may be called more than once by the SDK.

[nuno-vieira]

We recommend setting missing data to zero when creating QoSObject and it is handled by the SDK.

2020-11-02 20:09:17.344116+0000 ReferenceAppiOS_Helio[5423:5965472] [WARN] [2020-11-02 at 20:09:17][ADB_VHB_Heartbeat] #destroy() > Instance already destroyed. This log can be safely ignored. The destroy method frees some resources and may be called more than once by the SDK.

Okay we will do, but what happens if we don't set the missing data to 0 ?

UPDATE: We are already setting all values of QoS Object to 0 or empty string, as default values

[praveek]

@nuno-vieira The SDK holds a weak reference to the delegate passed to it. The getCurrentPlaybackTime and getQoSObject methods are called from a different thread. Can you check if the implementation is holding reference to delegate and the delegate implementation is thread safe and does not have any race conditions ?

FYI This crash https://github.com/Adobe-Marketing-Cloud/media-sdks/issues/51 was very similar and we fixed it by avoiding race conditions when accessing ad information from different threads.

[nuno-vieira]

@nuno-vieira The SDK holds a weak reference to the delegate passed to it. The getCurrentPlaybackTime and getQoSObject methods are called from a different thread. Can you check if the implementation is holding reference to delegate and the delegate implementation is thread safe and does not have any race conditions ?

FYI This crash #51 was very similar and we fixed it by avoiding race conditions when accessing ad information from different threads.

Yes the implementation is holding a reference to the delegate, or otherwise the HB wouldn't even work. Since the delegate has a weak reference, if we didn't had a strong reference to the delegate it would be automatically deallocated. Our delegate implementation is pretty simple, but because of how Kotlin/Native works, we need to update the values outside of the Delegate. What this means is that the delegate doesn't have a reference to the Player. But once the values are updated in the Player, we update it in the delegate. This maybe could cause some race conditions if we update these values and Adobe in another threads reads them. But because of how K/N works, we actually need to make sure these values are thread safe or they wouldn't even work. (This is a protection from K/N)

Note: K/N = Kotlin Native

Our delegate implementation:

internal class AdobeMediaHeartbeatDelegate() : NSObject(), ADBMediaHeartbeatDelegateProtocol {

    var currentBitrateInBps: AtomicReference<Double> = AtomicReference(0.0),
    var startupTimeInSeconds: AtomicReference<Double> = AtomicReference(0.0),
    var currentFps: AtomicReference<Double> = AtomicReference(0.0),
    var currentDroppedFrames: AtomicReference<Double> = AtomicReference(0.0),
    var currentPlaybackTimeInSeconds: AtomicReference<Double> = AtomicReference(0.0)

    override fun getQoSObject(): ADBMediaObject {
        return ADBMediaHeartbeat.createQoSObjectWithBitrate(
            bitrate = currentBitrateInBps.value,
            startupTime = startupTimeInSeconds.value,
            fps = currentFps.value,
            droppedFrames = currentDroppedFrames.value
        )
    }

    override fun getCurrentPlaybackTime(): NSTimeInterval {
        return currentPlaybackTimeInSeconds.value
    }
}

But if you guys had a similar issue on #51 and where able to fix it, can't it be the same issue? Couldn't you guys apply the same solution here?

[praveek]

In case of #51, we were able to fix it internally as the SDK holds the reference to the adObject. In case of cacheQoSInfo and cachePlayheadInfo the methods ends up calling the delegate implementation.

Looking at the call stack and comparing with code, I notice that the crash happens when the SDK tries to access the delegate (Line 11). I tested different scenarios with sample app but could not reproduce similar crash with obj-c/swift app. Does this seem familiar because Line 6-10 is implemented by K/N runtime and does the workflow follow threading model required by K/N apps?

1  libsystem_platform.dylib       0x2378e737c _os_unfair_lock_lock_slow + 272
2  libobjc.A.dylib                0x236ecc270 objc_object::sidetable_clearDeallocating() + 104
3  libobjc.A.dylib                0x236ec1700 objc_destructInstance + 140
4  libswiftCore.dylib             0x26530ecd4 swift_deallocClassInstance + 32
5  libswiftCore.dylib             0x26530ec68 _swift_release_dealloc + 28
6  AddonManager                   0x102049900 (anonymous namespace)::freeContainer(ContainerHeader*) + 316
7  AddonManager                   0x10204817c (anonymous namespace)::garbageCollect(MemoryState*, bool) + 512
8  AddonManager                   0x10204d364 ReleaseHeapRefStrict + 208
9  AddonManager                   0x10204ae3c BackRefFromAssociatedObject::tryAddRef() + 580
10 AddonManager                   0x102044f0c (anonymous namespace)::_tryRetainImp(objc_object*, objc_selector*) + 36
11 libobjc.A.dylib                0x236ecb7cc objc_loadWeakRetained + 352
12 AddonManager                   0x1020bb4b0 -[ADBMediaHeartbeat qosInfo] + 60```

[nuno-vieira]

I would expect that the workflow follows the K/N rules, because from our experience, when it doesn't, it crashes very easily. Even us trying to reproduce this in our reference app it is very hard, we did reproduce it already, but was very random and it was only 2 times. Even trying to reproduce and trying to run concurrent tests in this part of the code, we can't make it crash. We will keep investigating and maybe contact someone from Kotlin/Native and see if they can point us in the right direction. But all the information we can have it will be useful. Are these qosObjects stored in a reference somewhere? Or are they used and then discarded?

[praveek]

@nuno-vieira I looked into K/N runtime and see this commit related to the crash we see https://github.com/JetBrains/kotlin-native/commit/34cd218e4aa24eb9b7c890f5af5e9e80d80b17c9

Can you check this and verify if the version of K/N you use fixes this issue ?

[nuno-vieira]

@nuno-vieira I looked into K/N runtime and see this commit related to the crash we see

https://github.com/JetBrains/kotlin-native/commit/34cd218e4aa24eb9b7c890f5af5e9e80d80b17c9

Can you check this and verify if the version of K/N you use fixes this issue ?

Hi @praveek, nice catch! I'm pretty confident that is the problem. We are using K/N 1.4.10, so it includes this bug, and it is only fixed on K/N 1.4.30 which hasn't come out yet 😞 Unfortunately we can't downgrade K/N version to 1.3 because it took already a lot of work to update to K/N 1.4.

[tnspinks]

The follow message was sent to Adobe Customer Care in [Incident: 201030-001083] Adobe Media SDK crashes on Peacock TV apps:

Hello, an update on this ticket. We could use some help / guidance here.

The developers are trying to reproduce the issue that was raised by @Vieira, Nuno (Sky) here: https://github.com/Adobe-Marketing-Cloud/media-sdks/issues/62. It looks that the suggestion that it is a Kotlin Native (K/N) issue makes a lot of sense, but the team is struggling to reproduce the crash issue. The team is wondering if there is a version of the SDK that will aid us in testing/reproducing the issue? We are on version 2.7.7. An ideal fix would be a small change to expose the ADBMediaHeartbeatDelegate on the ADBMediaHeartbeat class, so that they can set and unset it. Is this possible?

[praveek]

@nuno-vieira @tnspinks Can you help me understand how adding an API would help in your case? The SDK currently only holds a weak reference inside ADBMediaHeartbeat class. If you need this API only for testing, i can provide a dev build with an API to set/unset ADBMediaHeartbeatDelegate.

[nuno-vieira]

@praveek If you look at the tests of that Kotlin/Native here: https://github.com/JetBrains/kotlin-native/commit/34cd218e4aa24eb9b7c890f5af5e9e80d80b17c9#diff-b18c1a740ab2c6abb3aef8955884626077368d5276745f61784b778156a2ef93R6-R25

We would like to replicate that test, to try to reproduce the issue. But without the access to the delegate, we can't really do it. Before migrating to the new Adobe Platform, we have a workaround that we want to try, but without replicating the crash it is harder to understand if it will fix anything, and only after we roll out a new version to the customers is that we will know if it fixed or not.

[theRealRobG]

Hi @praveek

We only need this API for testing, so a dev build will be welcome.

In the KN issue linked, you can see that the test added to ensure that the issue does not occur after the change, involves setting the weak object (analogous to the delegate) 100 times, to make the chance that the KN garbage collector collides with the Objective-C runtime more likely.

Before we try and fix this on our end, we want to have a relatively sure-fire way of replicating the crash, so that we can then determine if our fix actually works.

Therefore we believe that if we were able to take similar action to the KN test, by setting the delegate many times, while then calling methods on it (analogous to the loadManyTimes part), we would be more likely to see the crash.

[praveek]

Thanks @nuno-vieira and @theRealRobG. I will provide a dev build through customer support.