plu / pxctest

Execute tests in parallel on multiple iOS Simulators
MIT License
800 stars 57 forks source link

How to shutdown gracefully the simulators for headless testing? #16

Closed atineoSE closed 7 years ago

atineoSE commented 7 years ago

I am trying the headless testing with the boot-simulators command. It looks like new simulator devices are being created and booted in the provided folder (/tmp/test-simulators in your example).

How can I gracefully shutdown (and possibly delete) the newly created simulator devices after being done with the tests?

plu commented 7 years ago

This should work: xcrun simctl --set /tmp/test-simulators shutdown all

atineoSE commented 7 years ago

That worked, thanks!

okanerdogan09 commented 7 years ago

hi although i use this command i get the error below when i want to run the tests again. Do you have any other suggestion? Error Domain=com.facebook.FBSimulatorControl Code=0 "Failed to boot Simulator with options { env = { "SIMULATOR_IS_HEADLESS" = 1; }; persist = 1; "register-head-services" = 1; }" UserInfo={NSLocalizedDescription=Failed to boot Simulator with options { env = { "SIMULATOR_IS_HEADLESS" = 1; }; persist = 1; "register-head-services" = 1; }, NSUnderlyingError=0x7fdf7c469950 {Error Domain=com.apple.CoreSimulator.SimError Code=163 "Unable to boot device because it cannot be located on disk." UserInfo={NSLocalizedRecoverySuggestion=Use the device manager in Xcode or the simctl command line tool to either delete the device properly or erase contents and settings., NSLocalizedDescription=Unable to boot device because it cannot be located on disk., NSLocalizedFailureReason=The device's data is no longer present at /Users/user/mobile/ios-app_pxctest/ios-app/lounge/sim_devices/1/134EF32B-8B25-4592-8AA3-97EBE6029652/data.}}}

atineoSE commented 7 years ago

@okanerdogan84 can you post the exact command sequence you follow?

Here is a sample of mine:

# 1. Build for testing
xcodebuild build-for-testing \
-derivedDataPath /tmp/derivedData \
-project ./MyApp.xcodeproj \
-scheme MyAppTarget \
-sdk iphonesimulator \
-destination 'platform=iOS Simulator,name=iPhone 7 Plus'

...
** TEST BUILD SUCCEEDED **

# 2. Boot simulators
rm -rf /tmp/test-simulators
mkdir /tmp/test-simulators

pxctest boot-simulators \
--deviceset /tmp/test-simulators \
--destination 'name=iPhone 6,os=iOS 9.3' \
--destination 'name=iPad Air 2,os=iOS 10.2' \
--destination 'name=iPhone 7,os=iOS 10.2'

xcrun simctl --set /tmp/test-simulators list devices
== Devices ==
-- iOS 9.3 --
    iPhone 6 (E5E7A0B2-1410-470F-8AC9-51A54F2C0DCE) (Booted)
-- iOS 10.2 --
    iPhone 7 (6152C4AA-37EA-4F6E-AF3D-491E2645412F) (Booted)
    iPad Air 2 (2B4741F9-03C9-48A3-A1C8-F886476239EC) (Booted)
-- tvOS 10.1 --
-- watchOS 3.1 --

# 3. Run tests headlessly
pxctest run-tests \
--deviceset /tmp/test-simulators \
--destination 'name=iPhone 6,os=iOS 9.3' \
--destination 'name=iPad Air 2,os=iOS 10.2' \
--destination 'name=iPhone 7,os=iOS 10.2' \
--testrun /tmp/derivedData/Build/Intermediates/CodeCoverage/Products/MyAppTarget_iphonesimulator10.2-x86_64.xctestrun

...
Total - Finished executing 6 tests. 0 Failures, 0 Unexpected

xcrun simctl --set /tmp/test-simulators list devices
== Devices ==
-- iOS 9.3 --
    iPhone 6 (E5E7A0B2-1410-470F-8AC9-51A54F2C0DCE) (Booted)
-- iOS 10.2 --
    iPhone 7 (6152C4AA-37EA-4F6E-AF3D-491E2645412F) (Booted)
    iPad Air 2 (2B4741F9-03C9-48A3-A1C8-F886476239EC) (Booted)
-- tvOS 10.1 --
-- watchOS 3.1 --

# 4. Shut down devices
xcrun simctl --set /tmp/test-simulators shutdown all
xcrun simctl --set /tmp/test-simulators list devices
== Devices ==
-- iOS 9.3 --
    iPhone 6 (E5E7A0B2-1410-470F-8AC9-51A54F2C0DCE) (Shutdown)
-- iOS 10.2 --
    iPhone 7 (6152C4AA-37EA-4F6E-AF3D-491E2645412F) (Shutdown)
    iPad Air 2 (2B4741F9-03C9-48A3-A1C8-F886476239EC) (Shutdown)
-- tvOS 10.1 --
-- watchOS 3.1 --

# 5. Run tests again (this time the simulator window appears, i.e. tests are not executed headlessly but they still run)
pxctest run-tests \
--deviceset /tmp/test-simulators \
--destination 'name=iPhone 6,os=iOS 9.3' \
--destination 'name=iPad Air 2,os=iOS 10.2' \
--destination 'name=iPhone 7,os=iOS 10.2' \
--testrun /tmp/derivedData/Build/Intermediates/CodeCoverage/Products/MyAppTarget_iphonesimulator10.2-x86_64.xctestrun

...
Total - Finished executing 6 tests. 0 Failures, 0 Unexpected

xcrun simctl --set /tmp/test-simulators list devices
== Devices ==
-- iOS 9.3 --
    iPhone 6 (E5E7A0B2-1410-470F-8AC9-51A54F2C0DCE) (Booted)
-- iOS 10.2 --
    iPhone 7 (6152C4AA-37EA-4F6E-AF3D-491E2645412F) (Booted)
    iPad Air 2 (2B4741F9-03C9-48A3-A1C8-F886476239EC) (Booted)
-- tvOS 10.1 --
-- watchOS 3.1 --
okanerdogan09 commented 7 years ago

killall -9 com.apple.CoreSimulator.CoreSimulatorService worked for me. Thanks