Crash: [FUIArray insertSnapshot:withPreviousChildKey:]

victor-sarda commented 6 years ago

Describe your environment

Objective C or Swift: Swift
iOS version: 10.2.0
Firebase SDK version: 5.6.0
FirebaseUI version: 5.2.0
CocoaPods Version: 1.5.0

Describe the problem:

We got a crash report this morning that says

Fatal Exception: NSRangeException
*** -[__NSArrayM insertObject:atIndex:]: index 9223372036854775808 beyond bounds for empty array
-[FUIArray insertSnapshot:withPreviousChildKey:]

I was not able to reproduce this unfortunatly and it looked like a FirebaseUI to me and not something coming from our side. I might be wrong but I think those logs could help.

Steps to reproduce:

I could no reproduce this crash so far. I have a UIViewController with a tableview binded to a FUITableViewDataSource

Observed Results:

Here's what I've got from Crashlytics:

Fatal Exception: NSRangeException
*** -[__NSArrayM insertObject:atIndex:]: index 9223372036854775808 beyond bounds for empty array
-[FUIArray insertSnapshot:withPreviousChildKey:]

Fatal Exception: NSRangeException
0  CoreFoundation                 0x1923611b8 __exceptionPreprocess
1  libobjc.A.dylib                0x190d9855c objc_exception_throw
2  CoreFoundation                 0x19223fd44 CFStringConvertNSStringEncodingToEncoding
3  Bamboo                         0x1004fa318 -[FUIArray insertSnapshot:withPreviousChildKey:] (FUIArray.m:193)
4  Bamboo                         0x1004f9904 __24-[FUIArray observeQuery]_block_invoke (FUIArray.m:84)
5  Bamboo                         0x100326ee4 __43-[FChildEventRegistration fireEvent:queue:]_block_invoke.55 (FChildEventRegistration.m:74)
6  libdispatch.dylib              0x1911ea1fc _dispatch_call_block_and_release
7  libdispatch.dylib              0x1911ea1bc _dispatch_client_callout
8  libdispatch.dylib              0x1911eed68 _dispatch_main_queue_callback_4CF
9  CoreFoundation                 0x19230e810 __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__
10 CoreFoundation                 0x19230c3fc __CFRunLoopRun
11 CoreFoundation                 0x19223a2b8 CFRunLoopRunSpecific
12 GraphicsServices               0x193cee198 GSEventRunModal
13 UIKit                          0x1982817fc -[UIApplication _run]
14 UIKit                          0x19827c534 UIApplicationMain
15 Bamboo                         0x100090008 main (QueueTask.swift:26)
16 ???                            0x19121d5b8 (Missing)

Crashed: com.twitter.crashlytics.ios.exception
0x0000000000000000

Crashed: com.twitter.crashlytics.ios.exception
0  Bamboo                         0x10028f350 CLSProcessRecordAllThreads (CLSProcess.c:376)
1  Bamboo                         0x10028f810 CLSProcessRecordAllThreads (CLSProcess.c:407)
2  Bamboo                         0x10027f060 CLSHandler (CLSHandler.m:26)
3  Bamboo                         0x10028d960 __CLSExceptionRecord_block_invoke (CLSException.mm:199)
4  libdispatch.dylib              0x1911ea1bc _dispatch_client_callout + 16
5  libdispatch.dylib              0x1911f77f0 _dispatch_barrier_sync_f_invoke + 84
6  Bamboo                         0x10028d3d0 CLSExceptionRecord (CLSException.mm:206)
7  Bamboo                         0x10028d200 CLSExceptionRecordNSException (CLSException.mm:102)
8  Bamboo                         0x10028cdf8 CLSTerminateHandler() (CLSException.mm:259)
9  libc++abi.dylib                0x190d8766c std::__terminate(void (*)()) + 16
10 libc++abi.dylib                0x190d86f84 __cxxabiv1::exception_cleanup_func(_Unwind_Reason_Code, _Unwind_Exception*) + 134
11 libobjc.A.dylib                0x190d98690 _objc_exception_destructor(void*) + 362
12 CoreFoundation                 0x19223fd44 CFStringConvertNSStringEncodingToEncoding + 1134
13 Bamboo                         0x1004fa318 -[FUIArray insertSnapshot:withPreviousChildKey:] (FUIArray.m:193)
14 Bamboo                         0x1004f9904 __24-[FUIArray observeQuery]_block_invoke (FUIArray.m:84)
15 Bamboo                         0x100326ee4 __43-[FChildEventRegistration fireEvent:queue:]_block_invoke.55 (FChildEventRegistration.m:74)
16 libdispatch.dylib              0x1911ea1fc _dispatch_call_block_and_release + 24
17 libdispatch.dylib              0x1911ea1bc _dispatch_client_callout + 16
18 libdispatch.dylib              0x1911eed68 _dispatch_main_queue_callback_4CF + 1000
19 CoreFoundation                 0x19230e810 __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__ + 12
20 CoreFoundation                 0x19230c3fc __CFRunLoopRun + 1660
21 CoreFoundation                 0x19223a2b8 CFRunLoopRunSpecific + 444
22 GraphicsServices               0x193cee198 GSEventRunModal + 180
23 UIKit                          0x1982817fc -[UIApplication _run] + 684
24 UIKit                          0x19827c534 UIApplicationMain + 208
25 Bamboo                         0x100090008 main (QueueTask.swift:26)
26 ???                            0x19121d5b8 (Missing)

Expected Results:

I was expecting the table view to be updated and not to crash

victor-sarda commented 6 years ago

Hi 👋 This happened again today but I'm still not able to reproduce it. Here are the logs:

Fatal Exception: NSRangeException
*** -[__NSArrayM removeObjectsInRange:]: range {9223372036854775807, 1} extends beyond bounds for empty array
-[FUIArray removeSnapshot:withPreviousChildKey:]

Fatal Exception: NSRangeException
0  CoreFoundation                 0x18317ed8c __exceptionPreprocess
1  libobjc.A.dylib                0x1823385ec objc_exception_throw
2  CoreFoundation                 0x183117750 _CFArgv
3  CoreFoundation                 0x183105454 -[__NSArrayM removeObjectsInRange:]
4  Bamboo                         0x10345a44c -[FUIArray removeSnapshot:withPreviousChildKey:] (FUIArray.m:203)
5  Bamboo                         0x103459a7c __24-[FUIArray observeQuery]_block_invoke.58 (FUIArray.m:104)
6  Bamboo                         0x103286ee4 __43-[FChildEventRegistration fireEvent:queue:]_block_invoke.55 (FChildEventRegistration.m:74)
7  libdispatch.dylib              0x182a70b24 _dispatch_call_block_and_release
8  libdispatch.dylib              0x182a70ae4 _dispatch_client_callout
9  libdispatch.dylib              0x182a7d6e0 _dispatch_main_queue_callback_4CF$VARIANT$mp
10 CoreFoundation                 0x183127070 __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__
11 CoreFoundation                 0x183124bc8 __CFRunLoopRun
12 CoreFoundation                 0x183044da8 CFRunLoopRunSpecific
13 GraphicsServices               0x185027020 GSEventRunModal
14 UIKit                          0x18d02578c UIApplicationMain
15 Bamboo                         0x102ff0008 main (QueueTask.swift:26)
16 libdyld.dylib                  0x182ad5fc0 start

Crashed: com.twitter.crashlytics.ios.exception
0x0000000000000000
Crashed: com.twitter.crashlytics.ios.exception
0  Bamboo                         0x1031ef350 CLSProcessRecordAllThreads (CLSProcess.c:376)
1  Bamboo                         0x1031ef810 CLSProcessRecordAllThreads (CLSProcess.c:407)
2  Bamboo                         0x1031df060 CLSHandler (CLSHandler.m:26)
3  Bamboo                         0x1031ed960 __CLSExceptionRecord_block_invoke (CLSException.mm:199)
4  libdispatch.dylib              0x182a70ae4 _dispatch_client_callout + 16
5  libdispatch.dylib              0x182a79640 _dispatch_queue_barrier_sync_invoke_and_complete + 56
6  Bamboo                         0x1031ed3d0 CLSExceptionRecord (CLSException.mm:206)
7  Bamboo                         0x1031ed200 CLSExceptionRecordNSException (CLSException.mm:102)
8  Bamboo                         0x1031ecdf8 CLSTerminateHandler() (CLSException.mm:259)
9  libc++abi.dylib                0x18232937c std::__terminate(void (*)()) + 16
10 libc++abi.dylib                0x182328ccc __cxxabiv1::exception_cleanup_func(_Unwind_Reason_Code, _Unwind_Exception*) + 130
11 libobjc.A.dylib                0x182338720 _objc_exception_destructor(void*) + 362
12 CoreFoundation                 0x183117750 _CFArgv + 110
13 CoreFoundation                 0x183105454 -[__NSArrayM removeObjectsInRange:] + 2596
14 Bamboo                         0x10345a44c -[FUIArray removeSnapshot:withPreviousChildKey:] (FUIArray.m:203)
15 Bamboo                         0x103459a7c __24-[FUIArray observeQuery]_block_invoke.58 (FUIArray.m:104)
16 Bamboo                         0x103286ee4 __43-[FChildEventRegistration fireEvent:queue:]_block_invoke.55 (FChildEventRegistration.m:74)
17 libdispatch.dylib              0x182a70b24 _dispatch_call_block_and_release + 24
18 libdispatch.dylib              0x182a70ae4 _dispatch_client_callout + 16
19 libdispatch.dylib              0x182a7d6e0 _dispatch_main_queue_callback_4CF$VARIANT$mp + 1012
20 CoreFoundation                 0x183127070 __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__ + 12
21 CoreFoundation                 0x183124bc8 __CFRunLoopRun + 2272
22 CoreFoundation                 0x183044da8 CFRunLoopRunSpecific + 552
23 GraphicsServices               0x185027020 GSEventRunModal + 100
24 UIKit                          0x18d02578c UIApplicationMain + 236
25 Bamboo                         0x102ff0008 main (QueueTask.swift:26)
26 libdyld.dylib                  0x182ad5fc0 start + 4

morganchen12 commented 6 years ago

Do you ever invalidate your FUIArray?

morganchen12 commented 6 years ago

To clarify, the extremely large indexes here are NSNotFound values, which shouldn't appear since previousChildKey should always exist in the array when the insert/delete/move/change methods are called.

victor-sarda commented 6 years ago

Can you confirm dataSource.unbind() is invalidating the FUIArray ?

morganchen12 commented 6 years ago

It is. The arrays are supposed to be reusable after invalidation, though, so this is a bug.

morganchen12 commented 6 years ago

5.2.1 adds some diagnostics around this area. If the issue resurfaces, can you post the stack trace with exception message again?

victor-sarda commented 6 years ago

Yeah sure, no problem, I'll do that 👍

morganchen12 commented 5 years ago

Closing for inactivity, please comment again if the crash reappears.

DKucheryavyh commented 5 years ago

@morganchen12 I was able to reproduce very similar behavior with removeSnapshot and NSNotFound We have some Lambdas processing the database on childAdded event (and removing some of those if a condition is met).

We've traced it to a race condition, where some of the devices get those remove events before they get an actuall "add" event for that item (or those items were already moved out of the local snap). Maybe something similar applies to this case as well. The query is limited to the "last XX items", so the removed item might not be present in the Array.

Side note: Don't push the "Improved logging" into the master branch without public notes on the version release (if possible). Unexpected crashes on uncatchable @throw from inside some internal callback were somewhat unpleasant surprise.

DKucheryavyh commented 5 years ago

Exception Type:  EXC_CRASH (SIGABRT)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Exception Note:  EXC_CORPSE_NOTIFY
Triggered by Thread:  0

Last Exception Backtrace:
0   CoreFoundation                  0x1f39abef8 __exceptionPreprocess + 228 (NSException.m:172)
1   libobjc.A.dylib                 0x1f2b79a40 objc_exception_throw + 56 (objc-exception.mm:557)
2   Woosh                           0x1010f902c -[FUIArray removeSnapshot:withPreviousChildKey:] + 524 (FUIArray.m:220)
3   Woosh                           0x1010f843c __24-[FUIArray observeQuery]_block_invoke.58 + 100 (FUIArray.m:104)
4   Woosh                           0x1010786dc __43-[FChildEventRegistration fireEvent:queue:]_block_invoke.55 + 96 (FChildEventRegistration.m:74)
5   libdispatch.dylib               0x1f33e36c8 _dispatch_call_block_and_release + 24 (init.c:1372)
6   libdispatch.dylib               0x1f33e4484 _dispatch_client_callout + 16 (object.m:511)
7   libdispatch.dylib               0x1f33909ec _dispatch_main_queue_callback_4CF$VARIANT$mp + 1068 (inline_internal.h:2441)
8   CoreFoundation                  0x1f393a1bc __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__ + 12 (CFRunLoop.c:1813)
9   CoreFoundation                  0x1f3935084 __CFRunLoopRun + 1964 (CFRunLoop.c:3113)
10  CoreFoundation                  0x1f39345b8 CFRunLoopRunSpecific + 436 (CFRunLoop.c:3247)
11  GraphicsServices                0x1f5ba8584 GSEventRunModal + 100 (GSEvent.c:2245)
12  UIKitCore                       0x22092cbc8 UIApplicationMain + 212 (UIApplication.m:4341)

To be more specific, here is the stack trace. The self.snapshots doesn't contain specific key by the time the indexForKey:(NSString *)key is called. Database.database().reference().child(path).queryLimited(toLast: 20) - query. Database is updated frequently (both on the client side and by the cloud function to remove illegal records which have passed trough the client-side validation)

Unfortunately I can't provide a reliable way to reproduce the issue, since it really seems to be a race condition, since we have only 100 unique users trapped by it (out of 20+ thousands total).

It looks like the removeChild is fine to simply return without producing an exception in case the child is not found in the snapshots. Unless there is a way the child will be delivered somehow after the remove event.

Not sure how does the insertChild logic have to be altered to fix the race, though. May be some kind of cache to keep the orphaned objects until it's safe to remove/skip them?

firebase / FirebaseUI-iOS