aws-amplify / aws-sdk-ios

AWS SDK for iOS. For more information, see our web site:
https://aws-amplify.github.io/docs
Other
1.68k stars 885 forks source link

[AWSIoTDataManager publishString:onTopic:QoS:] crashes occasionally #4733

Closed babac87 closed 6 months ago

babac87 commented 1 year ago

Describe the bug

We are using AWSIoTDataManager to send the user's location every n seconds while the app is both in the foreground and background. We saw a similar app crashing pattern when network conditions changed (switching from WiFi to LTE and vice versa, losing network connection in a tunnel, etc.). We tried to simulate it with "Network Link Conditioner", but with no luck.

In stack trace it looks like last method called from AWS's SDK is -[AWSMQTTEncoder encodeWhenReady:], and after that, it crashes in SocketStream::write(__CFWriteStream*, unsigned char const*, long, CFStreamError*)

To Reproduce

As said above, it's a bit hard to reproduce. we saw a pattern when network conditions are changing.

Observed Behavior Expected Behavior

Some potential solutions would be to:

Stack Trace

Provided in screenshot

Code Snippet

manager.publishString(string, onTopic: topic, qoS: .messageDeliveryAttemptedAtMostOnce)

Areas of the SDK you are using (AWSMobileClient, Cognito, Pinpoint, IoT, etc)?

IoT specifically for this issue. We also use Cognito.

Screenshots

Screenshot 2023-05-11 at 14 54 55

Environment(please complete the following information):

Device Information (please complete the following information):

Additional context Add any other context about the problem here like your specific use case.

Relevant Console Output Logs

atierian commented 1 year ago

Thanks for reporting this @babac87

Please copy and paste the entire crash report in text form. Thanks!

babac87 commented 1 year ago

Hi @atierian,

Thanks for the response. Here is one of crash reports. I've changed name of the app to "XYZ".

Log:

Incident Identifier: 7A9209F8-81D6-46E3-9C7F-AA5EAD2E4637
Hardware Model:      iPhone13,3
Process:             XYZ [12627]
Path:                /private/var/containers/Bundle/Application/4671E70B-D767-4737-8C89-D8A192F656F2/XYZ.app/XYZ
Identifier:          network.XYZ.XYZ-PoC
Version:             1.0 (15)
AppStoreTools:       14E221
AppVariant:          1:iPhone13,3:16
Beta:                YES
Code Type:           ARM-64 (Native)
Role:                Non UI
Parent Process:      launchd [1]
Coalition:           network.XYZ.XYZ-PoC [8385]

Date/Time:           2023-05-22 22:05:52.6465 +0200
Launch Time:         2023-05-22 21:53:25.0147 +0200
OS Version:          iPhone OS 16.1.1 (20B101)
Release Type:        User
Baseband Version:    3.07.00
Report Version:      104

Exception Type:  EXC_BREAKPOINT (SIGTRAP)
Exception Codes: 0x0000000000000001, 0x00000001d0b04350
Termination Reason: SIGNAL 5 Trace/BPT trap: 5
Terminating Process: exc handler [12627]

Triggered by Thread:  0

Kernel Triage:
VM - pmap_enter retried due to resource shortage

Thread 0 name:
Thread 0 Crashed:
0   CoreFoundation                  0x00000001d0b04350 CFHash.cold.1 + 16 (CFRuntime.c:1287)
1   CoreFoundation                  0x00000001d09e30e0 CFHash + 732 (CFRuntime.c:1287)
2   CoreFoundation                  0x00000001d09bbf2c CFBasicHashGetCountOfKey + 216 (CFBasicHash.c:502)
3   CoreFoundation                  0x00000001d09beeb8 CFSetContainsValue + 176 (CFSet.c:229)
4   CoreFoundation                  0x00000001d0a42ccc CFRunLoopRemoveSource + 180 (CFRunLoop.c:3790)
5   CFNetwork                       0x00000001d1c1c374 SocketStream::write(__CFWriteStream*, unsigned char const*, long, CFStreamError*) + 604 (SocketStream.cpp:2968)
6   CoreFoundation                  0x00000001d09d111c CFWriteStreamWrite + 444 (CFStream.c:1242)
7   AWSIoT                          0x0000000103cd7910 -[AWSMQTTEncoder encodeWhenReady:] + 540 (AWSMQTTEncoder.m:144)
8   libdispatch.dylib               0x00000001d7f2dfdc _dispatch_client_callout + 20 (object.m:560)
9   libdispatch.dylib               0x00000001d7f3d574 _dispatch_lane_barrier_sync_invoke_and_complete + 56 (queue.c:1023)
10  AWSIoT                          0x0000000103cd76b8 -[AWSMQTTEncoder encodeMessage:] + 176 (AWSMQTTEncoder.m:94)
11  AWSIoT                          0x0000000103c65198 -[AWSMQTTSession send:] + 268 (AWSMQTTSession.m:675)
12  AWSIoT                          0x0000000103c6235c -[AWSMQTTSession publishDataAtMostOnce:onTopic:retain:] + 68 (AWSMQTTSession.m:195)
13  AWSIoT                          0x0000000103c5de9c -[AWSIoTMQTTClient publishData:qos:onTopic:retain:ackCallback:] + 720 (AWSIoTMQTTClient.m:870)
14  AWSIoT                          0x0000000103c5dafc -[AWSIoTMQTTClient publishString:qos:onTopic:] + 108 (AWSIoTMQTTClient.m:823)
15  AWSIoT                          0x0000000103ce6fd0 -[AWSIoTDataManager publishString:onTopic:QoS:] + 224 (AWSIoTDataManager.m:610)
16  XYZ                             0x0000000102cd8c74 AWSDataManager.send(data:) + 376 (AWSDataManager.swift:59)
17  XYZ                             0x0000000102d44cbc JourneyDataManager.push(data:) + 16 (JourneyDataManager.swift:30)
18  XYZ                             0x0000000102d44cbc protocol witness for DataManager.push(data:) in conformance JourneyDataManager + 20 (<compiler-generated>:29)
19  XYZ                             0x0000000102d44cbc JourneyManager.batchReady(_:) + 304 (JourneyManager.swift:75)
20  XYZ                             0x0000000102cbbe30 protocol witness for JourneyPacketManagerDelegate.batchReady(_:) in conformance JourneyManager + 4 (<compiler-generated>:0)
21  XYZ                             0x0000000102cbbe30 JourneyPacketManager.add(motion:) + 316
22  XYZ                             0x0000000102d384a0 JourneyManager.received(motion:) + 100 (JourneyManager.swift:67)
23  XYZ                             0x0000000102d384a0 protocol witness for MotionManagerDelegate.received(motion:) in conformance JourneyManager + 100 (<compiler-generated>:65)
24  XYZ                             0x0000000102d384a0 closure #1 in MotionManager.start() + 264 (MotionManager.swift:28)
25  XYZ                             0x0000000102d38558 thunk for @escaping @callee_guaranteed (@guaranteed CMDeviceMotion?, @guaranteed Error?) -> () + 80 (<compiler-generated>:0)
26  Foundation                      0x00000001caddb2f0 __NSBLOCKOPERATION_IS_CALLING_OUT_TO_A_BLOCK__ + 24 (NSOperation.m:1545)
27  Foundation                      0x00000001cadaf4a0 -[NSBlockOperation main] + 104 (NSOperation.m:1564)
28  Foundation                      0x00000001cadaf430 __NSOPERATION_IS_INVOKING_MAIN__ + 16 (NSOperation.m:2189)
29  Foundation                      0x00000001cad707d8 -[NSOperation start] + 708 (NSOperation.m:2206)
30  Foundation                      0x00000001cad7050c __NSOPERATIONQUEUE_IS_STARTING_AN_OPERATION__ + 16 (NSOperation.m:2220)
31  Foundation                      0x00000001cad75c20 __NSOQSchedule_f + 172 (NSOperation.m:2231)
32  libdispatch.dylib               0x00000001d7f3d114 _dispatch_block_async_invoke2 + 148 (queue.c:555)
33  libdispatch.dylib               0x00000001d7f2dfdc _dispatch_client_callout + 20 (object.m:560)
34  libdispatch.dylib               0x00000001d7f3c7f4 _dispatch_main_queue_drain + 928 (inline_internal.h:2632)
35  libdispatch.dylib               0x00000001d7f3c444 _dispatch_main_queue_callback_4CF + 44 (queue.c:7887)
36  CoreFoundation                  0x00000001d09ee6f8 __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__ + 16 (CFRunLoop.c:1780)
37  CoreFoundation                  0x00000001d09d0058 __CFRunLoopRun + 2036 (CFRunLoop.c:3147)
38  CoreFoundation                  0x00000001d09d4ed4 CFRunLoopRunSpecific + 612 (CFRunLoop.c:3418)
39  GraphicsServices                0x0000000209cd2368 GSEventRunModal + 164 (GSEvent.c:2196)
40  UIKitCore                       0x00000001d2eb33d0 -[UIApplication _run] + 888 (UIApplication.m:3745)
41  UIKitCore                       0x00000001d2eb3034 UIApplicationMain + 340 (UIApplication.m:5335)
42  SwiftUI                         0x00000001d44c0014 closure #1 in KitRendererCommon(_:) + 168 (UIKitApp.swift:37)
43  SwiftUI                         0x00000001d442116c runApp<A>(_:) + 152 (<compiler-generated>:0)
44  SwiftUI                         0x00000001d440a4bc static App.main() + 128 (App.swift:114)
45  XYZ                             0x0000000102caacc4 static XYZApp.$main() + 28 (XYZApp.swift:23)
46  XYZ                             0x0000000102caacc4 main + 40 (WalletTransaction.swift:0)
47  dyld                            0x00000001ef03c960 start + 2528 (dyldMain.cpp:1170)

Thread 1 name:
Thread 1:
0   libsystem_kernel.dylib          0x000000020d551b48 mach_msg2_trap + 8 (:-1)
1   libsystem_kernel.dylib          0x000000020d564008 mach_msg2_internal + 80 (mach_msg.c:201)
2   libsystem_kernel.dylib          0x000000020d564248 mach_msg_overwrite + 388 (mach_msg.c:0)
3   libsystem_kernel.dylib          0x000000020d55208c mach_msg + 24 (mach_msg.c:323)
4   CoreFoundation                  0x00000001d09ceaf0 __CFRunLoopServiceMachPort + 160 (CFRunLoop.c:2622)
5   CoreFoundation                  0x00000001d09cfd34 __CFRunLoopRun + 1232 (CFRunLoop.c:3005)
6   CoreFoundation                  0x00000001d09d4ed4 CFRunLoopRunSpecific + 612 (CFRunLoop.c:3418)
7   Foundation                      0x00000001cad76334 -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 212 (NSRunLoop.m:373)
8   Foundation                      0x00000001cad7621c -[NSRunLoop(NSRunLoop) runUntilDate:] + 64 (NSRunLoop.m:420)
9   UIKitCore                       0x00000001d2fe833c -[UIEventFetcher threadMain] + 436 (UIEventFetcher.m:1377)
10  Foundation                      0x00000001cad8f808 __NSThread__start__ + 716 (NSThread.m:963)
11  libsystem_pthread.dylib         0x000000021d9b46cc _pthread_start + 148 (pthread.c:893)
12  libsystem_pthread.dylib         0x000000021d9b3ba4 thread_start + 8 (:-1)

Thread 2 name:
Thread 2:
0   libsystem_kernel.dylib          0x000000020d551b48 mach_msg2_trap + 8 (:-1)
1   libsystem_kernel.dylib          0x000000020d564008 mach_msg2_internal + 80 (mach_msg.c:201)
2   libsystem_kernel.dylib          0x000000020d564248 mach_msg_overwrite + 388 (mach_msg.c:0)
3   libsystem_kernel.dylib          0x000000020d55208c mach_msg + 24 (mach_msg.c:323)
4   CoreFoundation                  0x00000001d09ceaf0 __CFRunLoopServiceMachPort + 160 (CFRunLoop.c:2622)
5   CoreFoundation                  0x00000001d09cfd34 __CFRunLoopRun + 1232 (CFRunLoop.c:3005)
6   CoreFoundation                  0x00000001d09d4ed4 CFRunLoopRunSpecific + 612 (CFRunLoop.c:3418)
7   CFNetwork                       0x00000001d1d2c5a8 +[__CFN_CoreSchedulingSetRunnable _run:] + 392 (CoreSchedulingSet.mm:1372)
8   Foundation                      0x00000001cad8f808 __NSThread__start__ + 716 (NSThread.m:963)
9   libsystem_pthread.dylib         0x000000021d9b46cc _pthread_start + 148 (pthread.c:893)
10  libsystem_pthread.dylib         0x000000021d9b3ba4 thread_start + 8 (:-1)

Thread 3 name:
Thread 3:
0   libsystem_kernel.dylib          0x000000020d551b48 mach_msg2_trap + 8 (:-1)
1   libsystem_kernel.dylib          0x000000020d564008 mach_msg2_internal + 80 (mach_msg.c:201)
2   libsystem_kernel.dylib          0x000000020d564248 mach_msg_overwrite + 388 (mach_msg.c:0)
3   libsystem_kernel.dylib          0x000000020d55208c mach_msg + 24 (mach_msg.c:323)
4   CoreFoundation                  0x00000001d09ceaf0 __CFRunLoopServiceMachPort + 160 (CFRunLoop.c:2622)
5   CoreFoundation                  0x00000001d09cfd34 __CFRunLoopRun + 1232 (CFRunLoop.c:3005)
6   CoreFoundation                  0x00000001d09d4ed4 CFRunLoopRunSpecific + 612 (CFRunLoop.c:3418)
7   CoreFoundation                  0x00000001d0a18d04 CFRunLoopRun + 64 (CFRunLoop.c:3444)
8   CoreMotion                      0x00000001dbef8ec0 CLMotionCore::runMotionThread(void*) + 1208 (CLMotionCore.mm:376)
9   libsystem_pthread.dylib         0x000000021d9b46cc _pthread_start + 148 (pthread.c:893)
10  libsystem_pthread.dylib         0x000000021d9b3ba4 thread_start + 8 (:-1)

Thread 4 name:
Thread 4:
0   libsystem_kernel.dylib          0x000000020d553480 kevent + 8 (:-1)
1   XYZ                             0x0000000103033424 realm::_impl::ExternalCommitHelper::listen() + 160 (external_commit_helper.cpp:219)
2   XYZ                             0x00000001030335b0 realm::_impl::ExternalCommitHelper::ExternalCommitHelper(realm::_impl::RealmCoordinator&, realm::RealmConfig const&)::$_0::operator()() const + 4 (external_commit_helper.cpp:175)
3   XYZ                             0x00000001030335b0 decltype(std::declval<realm::_impl::ExternalCommitHelper::ExternalCommitHelper(realm::_impl::RealmCoordinator&, realm::RealmConfig const&)::$_0>()()) std::__1::__invoke[abi:v15006]<realm::_impl::Ex... + 4 (invoke.h:394)
4   XYZ                             0x00000001030335b0 void std::__1::__thread_execute[abi:v15006]<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, realm::_impl::ExternalCommitHelper::ExternalCommit... + 4 (thread:290)
5   XYZ                             0x00000001030335b0 void* std::__1::__thread_proxy[abi:v15006]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, realm::_impl::ExternalCommitHelper:... + 56 (thread:301)
6   libsystem_pthread.dylib         0x000000021d9b46cc _pthread_start + 148 (pthread.c:893)
7   libsystem_pthread.dylib         0x000000021d9b3ba4 thread_start + 8 (:-1)

Thread 5 name:
Thread 5:
0   libsystem_kernel.dylib          0x000000020d5526ac select$DARWIN_EXTSN + 8 (:-1)
1   CoreFoundation                  0x00000001d0a1f6c4 __CFSocketManager + 636 (CFSocket.c:1340)
2   libsystem_pthread.dylib         0x000000021d9b46cc _pthread_start + 148 (pthread.c:893)
3   libsystem_pthread.dylib         0x000000021d9b3ba4 thread_start + 8 (:-1)

Thread 6:
0   libsystem_pthread.dylib         0x000000021d9b3b90 start_wqthread + 0 (:-1)

Thread 7:
0   libsystem_kernel.dylib          0x000000020d551b48 mach_msg2_trap + 8 (:-1)
1   libsystem_kernel.dylib          0x000000020d564008 mach_msg2_internal + 80 (mach_msg.c:201)
2   libsystem_kernel.dylib          0x000000020d564248 mach_msg_overwrite + 388 (mach_msg.c:0)
3   libsystem_kernel.dylib          0x000000020d55208c mach_msg + 24 (mach_msg.c:323)
4   CoreFoundation                  0x00000001d09ceaf0 __CFRunLoopServiceMachPort + 160 (CFRunLoop.c:2622)
5   CoreFoundation                  0x00000001d09cfd34 __CFRunLoopRun + 1232 (CFRunLoop.c:3005)
6   CoreFoundation                  0x00000001d09d4ed4 CFRunLoopRunSpecific + 612 (CFRunLoop.c:3418)
7   Foundation                      0x00000001cad76334 -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 212 (NSRunLoop.m:373)
8   AWSIoT                          0x0000000103c5d5e4 -[AWSIoTMQTTClient openStreams:] + 656 (AWSIoTMQTTClient.m:763)
9   Foundation                      0x00000001cad8f808 __NSThread__start__ + 716 (NSThread.m:963)
10  libsystem_pthread.dylib         0x000000021d9b46cc _pthread_start + 148 (pthread.c:893)
11  libsystem_pthread.dylib         0x000000021d9b3ba4 thread_start + 8 (:-1)

Thread 8:
0   libsystem_pthread.dylib         0x000000021d9b3b90 start_wqthread + 0 (:-1)

Thread 9:
0   libsystem_kernel.dylib          0x000000020d551b48 mach_msg2_trap + 8 (:-1)
1   libsystem_kernel.dylib          0x000000020d564008 mach_msg2_internal + 80 (mach_msg.c:201)
2   libsystem_kernel.dylib          0x000000020d564248 mach_msg_overwrite + 388 (mach_msg.c:0)
3   libsystem_kernel.dylib          0x000000020d55208c mach_msg + 24 (mach_msg.c:323)
4   CoreFoundation                  0x00000001d09ceaf0 __CFRunLoopServiceMachPort + 160 (CFRunLoop.c:2622)
5   CoreFoundation                  0x00000001d09cfd34 __CFRunLoopRun + 1232 (CFRunLoop.c:3005)
6   CoreFoundation                  0x00000001d09d4ed4 CFRunLoopRunSpecific + 612 (CFRunLoop.c:3418)
7   Foundation                      0x00000001cad76334 -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 212 (NSRunLoop.m:373)
8   AWSIoT                          0x0000000103c5d5e4 -[AWSIoTMQTTClient openStreams:] + 656 (AWSIoTMQTTClient.m:763)
9   Foundation                      0x00000001cad8f808 __NSThread__start__ + 716 (NSThread.m:963)
10  libsystem_pthread.dylib         0x000000021d9b46cc _pthread_start + 148 (pthread.c:893)
11  libsystem_pthread.dylib         0x000000021d9b3ba4 thread_start + 8 (:-1)

Thread 0 crashed with ARM Thread State (64-bit):
    x0: 0x0000000000000000   x1: 0x0000000000000000   x2: 0x0000000000000000   x3: 0x0000000fffffc088
    x4: 0x0000000000000000   x5: 0x0000000226fc03b0   x6: 0x0000000000000002   x7: 0x00000000ffffffff
    x8: 0x0000000227922000   x9: 0x00000001d0c721c4  x10: 0x0000000280d10200  x11: 0x00000001d0c63d28
   x12: 0x0000000000000001  x13: 0x0000000000000000  x14: 0x0000000000000002  x15: 0x00000001d1d5d34c
   x16: 0x0000000226fcfeb8  x17: 0x0000000226fcfeb8  x18: 0x0000000000000000  x19: 0x0000000283e7b090
   x20: 0x0000000000000000  x21: 0x0000000000000003  x22: 0x0000000226fd1c58  x23: 0x0000000280a32a40
   x24: 0x0000000280a32a50  x25: 0x0000000113dc4000  x26: 0x0000000280337960  x27: 0x0000000280414480
   x28: 0x000000022792a2e8   fp: 0x000000016d159c30   lr: 0x00000001d09e30e0
    sp: 0x000000016d159c10   pc: 0x00000001d0b04350 cpsr: 0x60001000
   esr: 0xf2000001 (Breakpoint) brk 1

Binary Images:
0x102ca4000 - 0x1034e3fff XYZ arm64  <58bd81a667df3c09a1606a6da449c5ca> /private/var/containers/Bundle/Application/4671E70B-D767-4737-8C89-D8A192F656F2/XYZ.app/XYZ
0x103c54000 - 0x103e9ffff AWSIoT arm64  <acacbb8b9acc34b7826a93e825839380> /private/var/containers/Bundle/Application/4671E70B-D767-4737-8C89-D8A192F656F2/XYZ.app/Frameworks/AWSIoT.framework/AWSIoT
0x1cad34000 - 0x1cb67dfff Foundation arm64e  <c431acb6fe043d28b6774de6e1c7d81f> /System/Library/Frameworks/Foundation.framework/Foundation
0x1d0954000 - 0x1d0d39fff CoreFoundation arm64e  <5cdc5d9ae5063740b64ebb30867b4f1b> /System/Library/Frameworks/CoreFoundation.framework/CoreFoundation
0x1d1ad6000 - 0x1d1e9dfff CFNetwork arm64e  <edb0559fc996327f9b3a6616e316f24d> /System/Library/Frameworks/CFNetwork.framework/CFNetwork
0x1d2b11000 - 0x1d42eefff UIKitCore arm64e  <179501b60fc2344ab969b4e3961ebe10> /System/Library/PrivateFrameworks/UIKitCore.framework/UIKitCore
0x1d42ef000 - 0x1d5c32fff SwiftUI arm64e  <494a7d5790af3e9096237038275b8d87> /System/Library/Frameworks/SwiftUI.framework/SwiftUI
0x1d7f2a000 - 0x1d7f70fff libdispatch.dylib arm64e  <fea36690a0003c55b7009120b05aa69b> /usr/lib/system/libdispatch.dylib
0x1dbee5000 - 0x1dc2d1fff CoreMotion arm64e  <84a0807ecd293c3cacb846549fef12f2> /System/Library/Frameworks/CoreMotion.framework/CoreMotion
0x1ef027000 - 0x1ef0aa00f dyld arm64e  <cb3ff411476234d286a4eca13f9fb6c3> /usr/lib/dyld
0x209cd1000 - 0x209cd9fff GraphicsServices arm64e  <a633a095122639428f413877660185ee> /System/Library/PrivateFrameworks/GraphicsServices.framework/GraphicsServices
0x20d551000 - 0x20d587ffb libsystem_kernel.dylib arm64e  <ff27fc8f90ba3332ab7ac5bc2d9ca7b1> /usr/lib/system/libsystem_kernel.dylib
0x21d9b3000 - 0x21d9befff libsystem_pthread.dylib arm64e  <1aa3a4b6f9e730568c8b4e4dd81312a4> /usr/lib/system/libsystem_pthread.dylib

EOF
babac87 commented 1 year ago

Additional info (don't know if it helps): we send around 50kB of data every 4 seconds.

atierian commented 1 year ago

Hey @babac87, I'm having trouble reproducing this. Have you been able reproduce the crash yourself?

babac87 commented 1 year ago

HI @atierian. Unfortunately, it still causes the app to crash. I've tried simulating with Network Link Conditioner, but with no luck. We also tried checking manager's connection, and sent to testers to test for a week, but it also crashed on network changes

if manager.getConnectionStatus() == .connected { 
    manager.publishString(string, onTopic: topic, qoS: .messageDeliveryAttemptedAtMostOnce)
}
Screenshot 2023-07-31 at 16 11 57
DiamondGlassDrill commented 1 year ago

Any news on this topic?

lawmicha commented 8 months ago

Hello, we released 2.33.9 which includes an IoT crash fix https://github.com/aws-amplify/aws-sdk-ios/pull/5185. Please upgrade to this and let us know if you are still experiencing the issue.

ruisebas commented 6 months ago

Closing due to inactivity. Please reopen if this issue still happens.