search

facebookarchive / xctool

An extension for Apple's xcodebuild that makes it easier to test iOS and macOS apps.
Apache License 2.0
6.91k stars 738 forks source link

Fix xctool crash on Mojave #759

Closed rmaz closed 5 years ago

rmaz commented 6 years ago

Remove a double close in the pipe processing code. This crashes in Mojave with the error:

BUG IN CLIENT OF LIBDISPATCH: Unexpected EV_VANISHED (do not destroy random mach ports or file descriptors)

From what I could see, these are occasionally closed twice eg in LaunchTaskAndCaptureOutput which closes the handles after task completion but before dispatch source cleanup. Given the ReadOutputsAndFeedOuputLinesToBlockOnQueue was not responsible for opening the pipes, it did not seem valid for this function to be closing them, this should be left to the caller.

shepting commented 5 years ago

@rmaz Do you not still get a crash when running scripts/make_release.sh on Mojave? Even on this branch I get:

Process:               xctool [98228]
Path:                  /private/var/folders/*/xctool
Identifier:            xctool
Version:               ???
Code Type:             X86-64 (Native)
Parent Process:        ??? [97349]
Responsible:           xctool [98228]
User ID:               510

Date/Time:             2018-12-06 10:23:04.174 -0800
OS Version:            Mac OS X 10.14.1 (18B75)
Report Version:        12
Anonymous UUID:        6D22972A-5F8D-C47D-32B8-C691D88D0304

Time Awake Since Boot: 150000 seconds

System Integrity Protection: enabled

Crashed Thread:        5

Exception Type:        EXC_BAD_INSTRUCTION (SIGILL)
Exception Codes:       0x0000000000000001, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Illegal instruction: 4
Termination Reason:    Namespace SIGNAL, Code 0x4
Terminating Process:   exc handler [98228]

Application Specific Information:
BUG IN CLIENT OF LIBDISPATCH: Unexpected EV_VANISHED (do not destroy random mach ports or file descriptors)

Thread 0:: Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib          0x00007fff6b72436a __ulock_wait + 10
1   libdispatch.dylib               0x00007fff6b59b5ac _dispatch_ulock_wait + 47
2   libdispatch.dylib               0x00007fff6b59bd43 _dispatch_group_wait_slow + 40
3   xctool                          0x000000010ed68852 -[RunTestsAction runTestables:options:xcodeSubjectInfo:] + 6060 (RunTestsAction.m:972)
4   xctool                          0x000000010ed66370 -[RunTestsAction performActionWithOptions:xcodeSubjectInfo:] + 1553 (RunTestsAction.m:586)
5   xctool                          0x000000010ed4ec81 -[XCTool run] + 3720 (XCTool.m:221)
6   xctool                          0x000000010ed4d3b8 main + 1508 (main.m:110)
7   xctool                          0x000000010ed48074 start + 52

Thread 1:: Dispatch queue: com.facebook.xctool.1544120569.092878.18
0   libsystem_kernel.dylib          0x00007fff6b722be6 close + 10
1   xctool                          0x000000010ed90390 __ReadOutputsAndFeedOuputLinesToBlockOnQueue_block_invoke_2.57 + 87 (TaskUtil.m:277)
2   libdispatch.dylib               0x00007fff6b59adcf _dispatch_client_callout + 8
3   libdispatch.dylib               0x00007fff6b5a6a2c _dispatch_lane_barrier_sync_invoke_and_complete + 60
4   xctool                          0x000000010ed8fc81 ReadOutputsAndFeedOuputLinesToBlockOnQueue + 908 (TaskUtil.m:280)
5   xctool                          0x000000010ed91173 LaunchTaskAndFeedSimulatorOutputAndOtestShimEventsToBlock + 606 (TaskUtil.m:428)
6   xctool                          0x000000010ed76ed0 -[OCUnitOSXLogicTestRunner runTestsAndFeedOutputTo:startupError:otherErrors:] + 326 (OCUnitOSXLogicTestRunner.m:122)
7   xctool                          0x000000010ed5803d -[OCUnitTestRunner runTests] + 479 (OCUnitTestRunner.m:209)
8   xctool                          0x000000010ed66fb3 __158-[RunTestsAction blockForTestable:focusedTestCases:allTestCases:testableExecutionInfo:testableTarget:isApplicationTest:arguments:environment:testRunnerClass:]_block_invoke + 512 (RunTestsAction.m:710)
9   xctool                          0x000000010ed6916a __56-[RunTestsAction runTestables:options:xcodeSubjectInfo:]_block_invoke.480 + 322 (RunTestsAction.m:944)
10  xctool                          0x000000010ed69384 __56-[RunTestsAction runTestables:options:xcodeSubjectInfo:]_block_invoke.502 + 90 (RunTestsAction.m:967)
11  libdispatch.dylib               0x00007fff6b599d53 _dispatch_call_block_and_release + 12
12  libdispatch.dylib               0x00007fff6b59adcf _dispatch_client_callout + 8
13  libdispatch.dylib               0x00007fff6b59d654 _dispatch_continuation_pop + 547
14  libdispatch.dylib               0x00007fff6b59cc7e _dispatch_async_redirect_invoke + 718
15  libdispatch.dylib               0x00007fff6b5a8d26 _dispatch_root_queue_drain + 325
16  libdispatch.dylib               0x00007fff6b5a94bd _dispatch_worker_thread2 + 90
17  libsystem_pthread.dylib         0x00007fff6b7da70e _pthread_wqthread + 619
18  libsystem_pthread.dylib         0x00007fff6b7da435 start_wqthread + 13

Thread 2:: Dispatch queue: xctool.runtests
0   libsystem_kernel.dylib          0x00007fff6b721c2a mach_msg_trap + 10
1   libsystem_kernel.dylib          0x00007fff6b722174 mach_msg + 60
2   com.apple.CoreFoundation        0x00007fff3e407da2 __CFRunLoopServiceMachPort + 337
3   com.apple.CoreFoundation        0x00007fff3e4072f1 __CFRunLoopRun + 1654
4   com.apple.CoreFoundation        0x00007fff3e406a28 CFRunLoopRunSpecific + 463
5   com.apple.Foundation            0x00007fff407b4cea -[NSConcreteTask waitUntilExit] + 220
6   xctool                          0x000000010ed914b9 __LaunchTaskAndFeedSimulatorOutputAndOtestShimEventsToBlock_block_invoke.137 + 31 (TaskUtil.m:423)
7   xctool                          0x000000010ed8fbdd ReadOutputsAndFeedOuputLinesToBlockOnQueue + 744
8   xctool                          0x000000010ed91173 LaunchTaskAndFeedSimulatorOutputAndOtestShimEventsToBlock + 606 (TaskUtil.m:428)
9   xctool                          0x000000010ed76ed0 -[OCUnitOSXLogicTestRunner runTestsAndFeedOutputTo:startupError:otherErrors:] + 326 (OCUnitOSXLogicTestRunner.m:122)
10  xctool                          0x000000010ed5803d -[OCUnitTestRunner runTests] + 479 (OCUnitTestRunner.m:209)
11  xctool                          0x000000010ed66fb3 __158-[RunTestsAction blockForTestable:focusedTestCases:allTestCases:testableExecutionInfo:testableTarget:isApplicationTest:arguments:environment:testRunnerClass:]_block_invoke + 512 (RunTestsAction.m:710)
12  xctool                          0x000000010ed6916a __56-[RunTestsAction runTestables:options:xcodeSubjectInfo:]_block_invoke.480 + 322 (RunTestsAction.m:944)
13  xctool                          0x000000010ed69384 __56-[RunTestsAction runTestables:options:xcodeSubjectInfo:]_block_invoke.502 + 90 (RunTestsAction.m:967)
14  libdispatch.dylib               0x00007fff6b599d53 _dispatch_call_block_and_release + 12
15  libdispatch.dylib               0x00007fff6b59adcf _dispatch_client_callout + 8
16  libdispatch.dylib               0x00007fff6b59d654 _dispatch_continuation_pop + 547
17  libdispatch.dylib               0x00007fff6b59cc7e _dispatch_async_redirect_invoke + 718
18  libdispatch.dylib               0x00007fff6b5a8d26 _dispatch_root_queue_drain + 325
19  libdispatch.dylib               0x00007fff6b5a94bd _dispatch_worker_thread2 + 90
20  libsystem_pthread.dylib         0x00007fff6b7da70e _pthread_wqthread + 619
21  libsystem_pthread.dylib         0x00007fff6b7da435 start_wqthread + 13
shepting commented 5 years ago

I'll try this on one of our CI machines though to see if it helps the exit code 132 issue on test flows requiring an App Host.

rmaz commented 5 years ago

The tests fail for me due to impossible sim configs or they deadlock, I don't see the above issue though. We have been running this version for a month without issue at this point (although the CI machines are not on Mojave yet, only developers, so not sure how useful that is).

shepting commented 5 years ago

Actually, @rmaz and @ExtremeMan when I test this on a real CI machine on Mojave it does work! It could just be something else about the tests that still needs fixing but this fixes our main issue!

ExtremeMan commented 5 years ago

@shepting thanks for debugging! I am happy to merge this. Though my concern is to fix this also on local machines. @shepting do you think you can help us to debug the issue?

rmaz commented 5 years ago

I think the issue is that the tests are not stable currently, and I don't believe it is related to this change. They are currently not even runnable on a Mojave machine due to outdated sims and runtimes, so I would suggest a separate change to get the tests fixed.

mgrebenets commented 5 years ago

Hi all, an eager user here curious when this PR will go in? 🤗 Thanks.

ExtremeMan commented 5 years ago

I pushed an alternative fix in https://github.com/facebook/xctool/commit/b893f431aeac888625a07321770a4bbb6337ce16. We don't close any fd's twice, at least I couldn't see where.

Can anyone try running on top of master and check if my fix works for them?