search

wix / Detox

Gray box end-to-end testing and automation framework for mobile apps
https://wix.github.io/Detox/
MIT License
11.23k stars 1.92k forks source link

Crash in DetoxSync #2641

Closed denissb closed 2 years ago

denissb commented 3 years ago

I have an issue when running tests in parallel with --workers 2 using jest-circus after upgrading to detox 18 The form tests that use the keyboard actions like typeText and clearText fail when they are running in parallel with another test.

Reproduction

  1. Have 2 tests, one that is verifying some on screen behaviour (fx. navigation or button actions) and one that is testing some form actions like entering and removing text in inputs to check form validation logic
  2. Run the tests in parallel with --workers 2 using jest-circus as a runner
  3. See the tests fail with random errors and a sync issue

I also get this error thrown in one of the tests when running in parallel

thrown: "Signal 11 was raised
    (
        0   Detox                               0x000000010aa97ef5 +[NSThread(DetoxUtils) dtx_demangledCallStackSymbols] + 37
        1   Detox                               0x000000010aa9b100 __DTXHandleCrash + 464
        2   Detox                               0x000000010aa9b845 __DTXHandleSignal + 59
        3   libsystem_platform.dylib            0x00007fff6115a5fd _sigtramp + 29
        4   ???                                 0x0000000000000000 0x0 + 0
        5   DetoxSync                           0x000000010b861aaf -[_DTXTimerTrampoline fire:] + 188
        6   DetoxSync                           0x000000010b849ef0 _DTXCFTimerTrampoline + 74
        7   CoreFoundation                      0x00007fff20390c57 __CFRUNLOOP_IS_CALLING_OUT_TO_A_TIMER_CALLBACK_FUNCTION__ + 20
        8   CoreFoundation                      0x00007fff2039072a __CFRunLoopDoTimer + 926
        9   CoreFoundation                      0x00007fff2038fcdd __CFRunLoopDoTimers + 265
        10  CoreFoundation                      0x00007fff2038a35e __CFRunLoopRun + 1949
        11  CoreFoundation                      0x00007fff203896d6 CFRunLoopRunSpecific + 567
        12  App                          0x00000001080f2fd1 +[RCTCxxBridge runRunLoop] + 993
        13  DetoxSync                           0x000000010b8523c0 swz_runRunLoopThread + 291
        14  Foundation                          0x00007fff20858e68 __NSThread__start__ + 1042
        15  libsystem_pthread.dylib             0x00007fff61167109 _pthread_start + 148
        16  libsystem_pthread.dylib             0x00007fff61162b8b thread_start + 15
    )"

Expected behaviour

The tests should pass just like when you run them with --maxWorkers 1

Environment (please complete the following information):

LeoNatan commented 3 years ago

This looks like a crash in DetoxSync. @alon-ha @d4vidi 

        4   ???                                 0x0000000000000000 0x0 + 0
        5   DetoxSync                           0x000000010b861aaf -[_DTXTimerTrampoline fire:] + 188
        6   DetoxSync                           0x000000010b849ef0 _DTXCFTimerTrampoline + 74

It could be a bad app code, but it could be a bug in DetoxSync.

LeoNatan commented 3 years ago

I don't think the number of workers matters here.

LeoNatan commented 3 years ago

Might be similar to #2627 (which was closed in error as duplicate of another issue).

denissb commented 3 years ago

I don't think the number of workers matters here.

Maybe it doesn't but when I run the tests without --workers 2 they are rock solid and pass constantly. I also didn't face any issues with the same test suit running on Detox 17.14.8 whatsoever.

LeoNatan commented 3 years ago

Detox 18 has a new synchronization mechanism, built in-house.

denissb commented 3 years ago

Yes, but it is quite nice. Thats why I want to upgrade, so now I am also digging around trying to find the cause. I will try to upgrade to 18.6.0 and see if it changes anything.

sam-barker commented 3 years ago

@denissb Please comment if you find the same crash when upgrading to the latest v18. When I last tried it happened on 18.3

d4vidi commented 3 years ago

@alon-ha any leads to help out?

denissb commented 3 years ago

@sam-barker I still have the same issue on 18.6.0

sam-barker commented 3 years ago

Any movement on this at all?

DanielMarkiel commented 3 years ago

Hi, I'm facing the same issue on my project but with --maxWorkers 1. All tests are passing fine on Android emulator. Tests were passing fine on iOS with Detox: 17.14.9.

Environment:

Logs:

thrown: "Signal 11 was raised
    (
        0   Detox                               0x000000010fec6eb5 +[NSThread(DetoxUtils) dtx_demangledCallStackSymbols] + 37
        1   Detox                               0x000000010feca0c0 __DTXHandleCrash + 464
        2   Detox                               0x000000010feca805 __DTXHandleSignal + 59
        3   libsystem_platform.dylib            0x00007fff6115cd7d _sigtramp + 29
        4   libsystem_c.dylib                   0x00007fff86d4a728 __global_locale + 0
        5   CoreFoundation                      0x00007fff20490049 -[__NSDictionaryM dealloc] + 128
        6   libobjc.A.dylib                     0x00007fff2018f834 objc_object::sidetable_release(bool, bool) + 174
        7   libobjc.A.dylib                     0x00007fff2018bb85 _object_remove_assocations + 580
        8   libobjc.A.dylib                     0x00007fff20188f25 objc_destructInstance + 84
        9   libobjc.A.dylib                     0x00007fff2018efb8 -[NSObject dealloc] + 21
        10  App                       0x000000010eac59a5 -[FBSDKGraphRequestConnection dealloc] + 96
        11  libobjc.A.dylib                     0x00007fff2018f834 objc_object::sidetable_release(bool, bool) + 174
        12  CFNetwork                           0x00007fff236d0b10 _CFNetworkHTTPConnectionCacheSetLimit + 163652
        13  CFNetwork                           0x00007fff23519ec5 CFNetwork + 28357
        14  DetoxSync                           0x0000000111d1b270 ____detox_sync_dispatch_wrapper_block_invoke + 23
        15  libdispatch.dylib                   0x00007fff2010532f _dispatch_call_block_and_release + 12
        16  libdispatch.dylib                   0x00007fff20106508 _dispatch_client_callout + 8
        17  libdispatch.dylib                   0x00007fff2010c3f9 _dispatch_lane_serial_drain + 715
        18  libdispatch.dylib                   0x00007fff2010cfab _dispatch_lane_invoke + 458
        19  libdispatch.dylib                   0x00007fff20117577 _dispatch_workloop_worker_thread + 782
        20  libsystem_pthread.dylib             0x00007fff61167499 _pthread_wqthread + 314
        21  libsystem_pthread.dylib             0x00007fff61166467 start_wqthread + 15
    )"

   4 |
      5 | describe('Intro', () => {
    > 6 |   beforeAll(async () => {
        |   ^
      7 |     await device.launchApp(deviceLaunchAppConfig);
      8 |     await device.setURLBlacklist(blacklistedURLs);
      9 |   });
stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe the issue is still relevant, please test on the latest Detox and report back.

Thank you for your contributions!

For more information on bots in this reporsitory, read this discussion.

d4vidi commented 3 years ago

All, we are currently on a break and a bit short-staffed at the moment, in general. We hope to get on top of this within 1-2 weeks. Here are similar discussions that could be relevant, in the meantime:

shamilovtim commented 3 years ago

Hello, any updates on this? We are sorely in need of a Firebase upgrade (the old version of Firebase is actually causing bugs in our app) but we can't bump Firebase until this gets resolved.

wdospinal commented 3 years ago

Hi, any advice on this, the app is having this issue when trying to add firebase and segment to the project

noomorph commented 3 years ago

@alon-ha, pinging you just for visibility.

Personally, I think this is an important issue since many RN users are using Firebase including our teams as well if I am not mistaken.

Let's check if we already have clear instructions on how to reproduce the crash. @wdospinal, @shamilovtim could you help to locate any demo project crashing with that error or create one?

I think Alon will get back to his job duties starting from the next week, and I hope he'll find time during April-May to check out a demo project if it will be already prepared.

shamilovtim commented 3 years ago

From what I could tell based on shallow research the issue might be caused by monkey patching in HTTP libraries. That's supposedly how it appeared in KTOR: https://github.com/firebase/firebase-ios-sdk/issues/7277

I've seen other tickets in the ecosystem also blaming the same thing in various ways

shamilovtim commented 3 years ago

Also related: https://github.com/JetBrains/kotlin-native/pull/4569 (blame was again monkey patching)

noomorph commented 3 years ago

@shamilovtim, what but isn't that Android-related there? At the moment, we are discussing iOS.

shamilovtim commented 3 years ago

I believe it was kotlin-native (compiles to iOS) so that's why it was iOS relevant. The underlying issue caused a bug in iOS code but it connected to kotlin code.

noomorph commented 3 years ago

@shamilovtim ah, I never realized that Firebase was using there kotlin-native, okay, cool. 👍

shamilovtim commented 3 years ago

Please see here: #270

Same error, signal 11, and similar stack trace that points to a swizzle.

There is a track record of this kind of swizzling / reflection causing issues and breaking Firebase. I assume the fix is probably pretty similar

sam-barker commented 3 years ago

Is there any update on this? I assume a fair few projects are a major version behind now and still on v17 because of this issue. I know we certainly are

shamilovtim commented 3 years ago

We've having to work around this in a bad way. Other packages that were depending on Firebase were falling behind to the point of becoming show stoppers. Firebase is not optional in our project, multiple other packages depend on Firebase and Detox is not working with Firebase.

shamilovtim commented 3 years ago

More logs:

Crash 1:

2021-05-17 13:50:49.509 I  Shipt[30146:a0e86] [com.apple.network:connection] nw_endpoint_handler_cancel [C27 localhost:32301 ready resolver (satisfied (Path is satisfied), interface: en0)]
2021-05-17 13:50:49.509 I  Shipt[30146:a0e86] [com.apple.network:connection] nw_association_schedule_deactivation <nw_association localhost:32301> will become dormant after 10000ms of inactivity
2021-05-17 13:50:49.509 I  Shipt[30146:a0e86] [com.apple.network:connection] nw_endpoint_handler_cancel [C27.1 127.0.0.1:32301 ready socket-flow (satisfied (Path is satisfied), interface: lo0)]
2021-05-17 13:50:49.509 I  Shipt[30146:a0e86] [com.apple.network:connection] nw_association_schedule_deactivation <nw_association 127.0.0.1:32301> will become dormant after 10000ms of inactivity
2021-05-17 13:50:49.509 Df Shipt[30146:a0e86] [com.apple.network:connection] nw_flow_disconnected [C27.1 127.0.0.1:32301 cancelled socket-flow ((null))] Output protocol disconnected
2021-05-17 13:50:49.509 Db Shipt[30146:a0e86] [com.apple.network:connection] nw_resolver_cancel [C27] 0x7ffa0a538060
2021-05-17 13:50:49.509 Df Shipt[30146:a0e86] [com.apple.network:connection] nw_connection_report_state_with_handler_on_nw_queue [C27] reporting state cancelled
2021-05-17 13:50:49.510 I  Shipt[30146:a0e86] [com.apple.network:connection] [C27 localhost:32301 tcp, url hash: 08d3ccea] dealloc
2021-05-17 13:50:49.510 Df Shipt[30146:a0e86] (CFNetwork) Connection 27: done
2021-05-17 13:50:49.513 Db Shipt[30146:a0e86] [com.apple.CFNetwork:Coalescing] removing all entries config 0x60000052f020
2021-05-17 13:50:49.513 Df Shipt[30146:a0e86] (CFNetwork) Connection 33: cleaning up
2021-05-17 13:50:49.513 Df Shipt[30146:a0e86] [com.apple.network:connection] [C33 37907897-6175-411B-BE0F-58F6A5AEAC16 localhost:8081 tcp, url hash: f6991239] cancel
2021-05-17 13:50:49.513 Df Shipt[30146:a0e86] [com.apple.network:connection] [C33 37907897-6175-411B-BE0F-58F6A5AEAC16 localhost:8081 tcp, url hash: f6991239] cancelled
    [C33.1 4C68FBA4-F704-42B5-96F4-6C31EF693AE2 127.0.0.1:65467<->127.0.0.1:8081]
    Connected Path: satisfied (Path is satisfied), viable, interface: lo0
    Duration: 0.249s, DNS @0.001s took 0.003s, TCP @0.006s took 0.001s
    bytes in/out: 16650029/247, packets in/out: 1090/1, rtt: 0.000s, retransmitted bytes: 0, out-of-order bytes: 0
2021-05-17 13:50:49.513 I  Shipt[30146:a0e86] [com.apple.network:connection] nw_endpoint_handler_cancel [C33 localhost:8081 ready resolver (satisfied (Path is satisfied), interface: en0)]
2021-05-17 13:50:49.513 I  Shipt[30146:a0e86] [com.apple.network:connection] nw_association_schedule_deactivation <nw_association localhost:8081> will become dormant after 10000ms of inactivity
2021-05-17 13:50:49.513 I  Shipt[30146:a0e86] [com.apple.network:connection] nw_endpoint_handler_cancel [C33.1 127.0.0.1:8081 ready socket-flow (satisfied (Path is satisfied), interface: lo0)]
2021-05-17 13:50:49.513 I  Shipt[30146:a0e86] [com.apple.network:connection] nw_association_schedule_deactivation <nw_association 127.0.0.1:8081> will become dormant after 10000ms of inactivity
2021-05-17 13:50:49.513 Df Shipt[30146:a0e86] [com.apple.network:connection] nw_flow_disconnected [C33.1 127.0.0.1:8081 cancelled socket-flow ((null))] Output protocol disconnected
2021-05-17 13:50:49.513 Db Shipt[30146:a0e86] [com.apple.network:connection] nw_resolver_cancel [C33] 0x7ff9ea507820
2021-05-17 13:50:49.514 Df Shipt[30146:a0e86] [com.apple.network:connection] nw_connection_report_state_with_handler_on_nw_queue [C33] reporting state cancelled
2021-05-17 13:50:49.514 E  Shipt[30146:a0e87] [com.wix.Detox:DetoxManager] App crashed: Signal 11 was raised
(
    0   Detox                               0x0000000111515ef5 +[NSThread(DetoxUtils) dtx_demangledCallStackSymbols] + 37
    1   Detox                               0x0000000111519100 __DTXHandleCrash + 464
    2   Detox                               0x0000000111519845 __DTXHandleSignal + 59
    3   libsystem_platform.dylib            0x00007fff60335d7d _sigtramp + 29
    4   ???                                 0x0000700002bc6580 0x0 + 123145348212096
    5   CoreFoundation                      0x00007fff20492833 -[__NSDictionaryM dealloc] + 128
    6   libobjc.A.dylib                     0x00007fff20191604 objc_object::sidetable_release(bool, bool) + 174
    7   libobjc.A.dylib                     0x00007fff2018d8ad _object_remove_assocations + 562
    8   libobjc.A.dylib                     0x00007fff2018ac02 objc_destructInstance + 84
    9   libobjc.A.dylib                     0x00007fff20190d88 -[NSObject dealloc] + 21
    10  libobjc.A.dylib                     0x00007fff20191604 obj<…>
2021-05-17 13:50:49.514 I  Shipt[30146:a0e87] [com.wix.Detox:DetoxInstrumentsManager] Called stop but no recording in progress
2021-05-17 13:50:49.514 I  Shipt[30146:a0e86] [com.apple.network:connection] [C33 localhost:8081 tcp, url hash: f6991239] dealloc
2021-05-17 13:50:49.522 I  Shipt[30146:a0d10] [com.wix.Detox:WebSocket] Action received: reactNativeReload

Crash two:

2021-05-17 14:10:42.271 Df Shipt[30786:a7ebb] [com.apple.network:connection] nw_flow_disconnected [C29.1 172.217.4.42:443 cancelled socket-flow ((null))] Output protocol disconnected
2021-05-17 14:10:42.271 I  Shipt[30786:a7eca] [com.apple.runningboard:assertion] Acquiring assertion: <RBSAssertionDescriptor| "Shared Background Assertion 3 for com.shipt.groceries-staging" ID:(null) target:30786>
2021-05-17 14:10:42.271 Db Shipt[30786:a7ebb] [com.apple.network:connection] nw_resolver_cancel [C29] 0x7fb046c2ff60
2021-05-17 14:10:42.271 Df Shipt[30786:a7ebb] [com.apple.network:connection] nw_connection_report_state_with_handler_on_nw_queue [C29] reporting state cancelled
2021-05-17 14:10:42.271 Db Shipt[30786:a7ebb] [com.apple.CFNetwork:Coalescing] removing all entries config 0x600001ee6080
2021-05-17 14:10:42.271 Db Shipt[30786:a7ebb] [com.apple.network:boringssl] boringssl_bio_destroy(196)
2021-05-17 14:10:42.272 Db Shipt[30786:a7eca] [com.apple.runningboard:assertion] Adding assertion 24911-30786-1889 to dictionary
2021-05-17 14:10:42.272 I  Shipt[30786:a7ebb] [com.apple.network:connection] [C29 firebaselogging-pa.googleapis.com:443 tcp, url hash: cdff6ee4, tls] dealloc
2021-05-17 14:10:42.272 I  Shipt[30786:a7ebb] [com.apple.network:] nw_context_purge_endpoints Context <nw_context com.apple.CFNetwork.NSURLSession.{18F6C993-251C-4F4C-86F8-4BD2F4532047}{(null)}{Y}{2}> has more than 0 cache entries, purging 2 from 2 down to 0
2021-05-17 14:10:42.277 E  Shipt[30786:a7f28] [com.wix.Detox:DetoxManager] App crashed: Signal 11 was raised
(
    0   Detox                               0x0000000110710ef5 +[NSThread(DetoxUtils) dtx_demangledCallStackSymbols] + 37
    1   Detox                               0x0000000110714100 __DTXHandleCrash + 464
    2   Detox                               0x0000000110714845 __DTXHandleSignal + 59
    3   libsystem_platform.dylib            0x00007fff60335d7d _sigtramp + 29
    4   ???                                 0x000070000b3c1500 0x0 + 123145490797824
    5   CoreFoundation                      0x00007fff20492833 -[__NSDictionaryM dealloc] + 128
    6   libobjc.A.dylib                     0x00007fff20191604 objc_object::sidetable_release(bool, bool) + 174
    7   libobjc.A.dylib                     0x00007fff2018d8ad _object_remove_assocations + 562
    8   libobjc.A.dylib                     0x00007fff2018ac02 objc_destructInstance + 84
    9   libobjc.A.dylib                     0x00007fff20190d88 -[NSObject dealloc] + 21
    10  Foundation                          0x00007fff207f7ca1 -[N<…>
2021-05-17 14:10:42.277 I  Shipt[30786:a7f28] [com.wix.Detox:DetoxInstrumentsManager] Called stop but no recording in progress
2021-05-17 14:10:42.299 A  Shipt[30786:a7ebb] (Security) SecItemDelete_ios
2021-05-17 14:10:42.300 A  Shipt[30786:a7ebb] (Security) SecItemDelete_ios
2021-05-17 14:10:42.301 A  Shipt[30786:a7ebb] (Security) SecItemDelete_ios
2021-05-17 14:10:42.487 I  Shipt[30786:a7e81] [com.apple.UIKit:Application] Deactivation reason added: 12; deactivation reasons: 0 -> 4096; animating application lifecycle event: 0
shamilovtim commented 3 years ago

I have isolated the issue to UIApplicationDelegate swizzling. This bug is probably related to both detox and firebase swizzling UIApplicationDelegate

d4vidi commented 3 years ago

@alon-ha this could probably give us a good lead as to where the root of the problem is.

alon-ha commented 3 years ago

@shamilovtim Sorry for the delay. Can you please open an example project with both firebase and detox so I will try to solve it?

shamilovtim commented 3 years ago

If you have a default starter template you could point me to where you have built out some e2e tests I think it would just be a matter of adding Firebase to it, which I could do no problem. Then that could live on for debugging any future Firebase issues if/when they arise

alon-ha commented 3 years ago

We currently don't have a starter template. I'm working on one exactly for such cases

ball-hayden commented 3 years ago

Cross-posting here from https://github.com/wix/DetoxSync/issues/5#issuecomment-865851840

I've put together a minimal reproduction with Firebase Perf enabled: https://github.com/PlayerData/DetoxFirebaseRepro Worth noting that produces a different trace to that originally reported in this issue - I think there might be multiple issues here that have been combined together.

alon-ha commented 3 years ago

@ball-hayden That's great, I will try to look on the example you created. I created a general template which will help speed bugs like this which we need to reproduce locally. So next time use DetoxTemplate :)

alon-ha commented 3 years ago

@ball-hayden actually in the readme I see that you wrote that we need to setup a firebase project and some plist file.. That's exactly the part I want to skip on and just start to debug it from a point which firebase already setup and the tests failed because of that.. Will be great if you can fork the DetoxTemplate and do so (just follow the readme there).

ball-hayden commented 3 years ago

@alon-ha I'm really sorry - I can't share a private Firebase key to do that (you and the rest of the world would have access to do potentially nasty things with our Firebase account). The only way to reproduce this is to set up a Firebase project through their UI and drop in the magic plist file. I can provide more detailed steps for that if that would help?

alon-ha commented 3 years ago

@ball-hayden I didn't expect you to share your private key, absolutely agree with you, never do so. But I want you (or anyone which suffer from this bug) to set up some new project in Firebase and integrate to the DetoxTemplate, all can be done using some "fake" new account in firebase. Many issues here so I want to start with something which already reproduce, that's why I created this template.

ball-hayden commented 3 years ago

I don't think that would be possible @alon-ha. The missing plist file contains only API keys, and is downloaded from the Firebase Console. I've just tried populating that with fake details and the app fails to start - I assume the Firebase library is validating on startup.

I can fork DetoxTemplate, but after I have added React Native Firebase you would still need to create a Firebase "project" and download API keys.

If you aren't able to carry out steps 1-3 of https://firebase.google.com/docs/ios/setup (which I understand is a pain, but is only clicking through), I can privately send you the plist file for my test Firebase project?

alon-ha commented 3 years ago

I don't think that would be possible @alon-ha. The missing plist file contains only API keys, and is downloaded from the Firebase Console. I've just tried populating that with fake details and the app fails to start - I assume the Firebase library is validating on startup.

I can fork DetoxTemplate, but after I have added React Native Firebase you would still need to create a Firebase "project" and download API keys.

If you aren't able to carry out steps 1-3 of https://firebase.google.com/docs/ios/setup (which I understand is a pain, but is only clicking through), I can privately send you the plist file for my test Firebase project?

I meant that you can open a new account in firebase and set all those things into a fork of DetoxTemplate. Basically so the fork repo will already include everything.

ball-hayden commented 3 years ago

Unfortunately, I can't do that as that would still be sharing API keys publically (albeit for a test firebase account).

I'll get a fork ready. Either someone else who is willing to publish keys can do so, or we can exchange keys privately.

shamilovtim commented 3 years ago

They don't have accounts for OSS? Seems like something like that should exist

ball-hayden commented 3 years ago

@alon-ha my apologies - I've done a bit more reading into the GoogleService-Info.plist file and the contents are unique but considered to be non-secret.

I've pushed one to my original demo project. I'll also get a fork of the DetoxTemplate fully setup.

ball-hayden commented 3 years ago

@shamilovtim you'd need a free account to reproduce this - all of the necessary bits are in the free "spark" plan. So, yes, they kinda do have accounts for OSS.

ball-hayden commented 3 years ago

https://github.com/PlayerData/DetoxFirebaseReproTemplate ^ There's a fork with Firebase setup.

The issue is intermittent - I've adapted your provided test to run multiple times. I reliably hit the bug within the first few iterations using npm run test:ios-debug

alon-ha commented 3 years ago

@ball-hayden Thank you for the example. I'm hoping to start working on that this week. I already checked the swizzling in detox and seems that it was done correctly. Hopefully I can fix it in our side, but my best solution might be to open a PR to Firebase. Will keep you guys update

alon-ha commented 3 years ago

@ball-hayden I debugged the Template you provided. Unfortunately you can't put the describe inside a for loop, that's not a valid way to run e2e tests - the for loop won't wait for each to finish.. This will fail without Firebase as well (you can try yourself). Please write a test/senario which without Firebase configure line it will pass and fail with Firebase. Waiting for you to fix/add test which will demonstrate the actual problem

ball-hayden commented 3 years ago

Interesting. Is it possible that this bug is aggravated by Firebase, but does also exist without?

I've got to tap out for a bit, but I'll happily accept a PR to https://github.com/PlayerData/DetoxFirebaseReproTemplate from anyone else following this thread.

Otherwise, I'll update that repo when I've got some time available again.

I assume that simply duplicating the test file (instead of running in a loop) isn't enough to trigger the crash?

alon-ha commented 3 years ago

Is it possible that this bug is aggravated by Firebase, but does also exist without

Everything is possible (: But running a describe block inside a loop won't work regardless.. So I need a valid example which reproduce the bug. As you assumed simply duplicating the test file won't trigger a crash. The example should be something that without [FIRApp configure]; will probably work right but crash with it

shamilovtim commented 3 years ago

If you have enough tests in the project you'll be more guaranteed to hit the bug from what I can tell

alon-ha commented 3 years ago

If you have enough tests in the project you'll be more guaranteed to hit the bug from what I can tell

I disagree. Inside the repo we have a test project and each time detox version build on master, 33 tests files are running with an average of 3 test for each file.. So it's clearly not an issue of enough tests. That's exactly why I want to see a template with firebase which I can reproduce and debug it (clearly an issue with some swizzling), but I need to see it locally in order to help here

ball-hayden commented 3 years ago

I've pushed a change to https://github.com/PlayerData/DetoxFirebaseReproTemplate which uses multiple files rather than a for loop, and it looks like duplicating the test file does trigger the crash (admittedly, you have to duplicate it a few times...)

I've run the following:

npm run build:ios-release
npm run test:ios-release -- --workers 2

I'd didn't get a summary line from the output (for some reason), but I observe 5 fail with a Signal 11.

Sample Signal 11

 FAIL  e2e/ExampleScreen41.e2e.js (34.525 s)
  Example Screen Tests
    ✕ should type into the text field (5055 ms)

  ● Example Screen Tests › should type into the text field

    DetoxRuntimeError: The pending request #3 ("invoke") has been rejected due to the following error:

    The app has crashed, see the details below:

    Signal 11 was raised
    (
        0   Detox                               0x00000001040e0c45 +[NSThread(DetoxUtils) dtx_demangledCallStackSymbols] + 37
        1   Detox                               0x00000001040e3e50 __DTXHandleCrash + 464
        2   Detox                               0x00000001040e4595 __DTXHandleSignal + 59
        3   libsystem_platform.dylib            0x00007fff60335d7d _sigtramp + 29
        4   ???                                 0xc766036a6e348a5b 0x0 + -4078568656887051685
        5   CoreFoundation                      0x00000001070d5833 -[__NSDictionaryM dealloc] + 128
        6   libobjc.A.dylib                     0x0000000106f07604 objc_object::sidetable_release(bool, bool) + 174
        7   libobjc.A.dylib                     0x0000000106f038ad _object_remove_assocations + 562
        8   libobjc.A.dylib                     0x0000000106f00c02 objc_destructInstance + 84
        9   libobjc.A.dylib                     0x0000000106f06d88 -[NSObject dealloc] + 21
        10  Foundation                          0x0000000104dcdca1 -[NSOperation dealloc] + 281
        11  libobjc.A.dylib                     0x0000000106f07604 objc_object::sidetable_release(bool, bool) + 174
        12  libsystem_blocks.dylib              0x0000000109cb5432 _Block_release + 128
        13  libsystem_blocks.dylib              0x0000000109cb5432 _Block_release + 128
        14  libsystem_blocks.dylib              0x0000000109cb5432 _Block_release + 128
        15  DetoxTemplate                       0x000000010385dae6 __destroy_helper_block_e8_32s40s48s + 24
        16  libsystem_blocks.dylib              0x0000000109cb5432 _Block_release + 128
        17  libsystem_blocks.dylib              0x0000000109cb5432 _Block_release + 128
        18  libdispatch.dylib                   0x0000000109b7c8df _dispatch_client_callout + 8
        19  libdispatch.dylib                   0x0000000109b82f5c _dispatch_lane_serial_drain + 1042
        20  libdispatch.dylib                   0x0000000109b8398c _dispatch_lane_invoke + 400
        21  libdispatch.dylib                   0x0000000109b8df81 _dispatch_workloop_worker_thread + 772
        22  libsystem_pthread.dylib             0x00007fff6034045d _pthread_wqthread + 314
        23  libsystem_pthread.dylib             0x00007fff6033f42f start_wqthread + 15
    )
ball-hayden commented 3 years ago

Unfortunately you can't put the describe inside a for loop, that's not a valid way to run e2e tests - the for loop won't wait for each to finish..

I feel it might be helpful to pull this thread a bit. This was previously written as a synchronous for loop, not an async forEach block - why would the loop not wait for each to finish?