search

leancodepl / patrol

Flutter-first UI testing framework. Ready for action!
https://patrol.leancode.co
Apache License 2.0
910 stars 142 forks source link

Can't run Patrol tests on non-latest iOS version #1441

Open eroshinn opened 1 year ago

eroshinn commented 1 year ago

Hello!

First of all, thanks a lot for an awesome framework, it adds crucial missing features to the Flutter integration tests thus making our lives much easier

Steps to reproduce

1) Create a default Flutter project 2) Integrate Patrol, use the default example_test.dart 3) Launch any iOS simulator with non-latest iOS version (e.g. 15.5) 4) Try running a Patrol test

Actual results

Test doesn't launch, this error log is shown:

No device specified, using the first one (iPhone 13 Pro)
• Building app with entrypoint test_bundle.dart for iOS simulator (debug)...
✓ Completed building app with entrypoint test_bundle.dart for iOS simulator (26.2s)
• Running app with entrypoint test_bundle.dart for iOS simulator on simulator iPhone 13 Pro...
    2023-06-30 16:36:00.680 xcodebuild[70063:17854795] Writing error result bundle to /var/folders/5x/_dlphwk153z51v3xp2l30lccqn9pft/T/ResultBundle_2023-30-06_16-36-0000.xcresult
    xcodebuild: error: Unable to find a destination matching the provided destination specifier:
            { platform:iOS Simulator, OS:latest, name:iPhone 13 Pro }
...

The OS:latest bit seems suspicious: you can run a test without any problems if you use a simulator with the highest available iOS version your xcode allows

It seems like a bug, because Patrol should work fine with iOS ^13.0

Patrol version

patrol: ^1.1.2

Patrol Doctor output

Patrol Doctor output ``` You're using Patrol CLI 2.0, which has breaking changes. Read the migration guide at https://patrol.leancode.co/v2. Disable this warning by setting the PATROL_MIGRATED environment variable. Patrol CLI version: 2.0.0 Program adb not found Env var $ANDROID_HOME is not set Program xcodebuild found in /usr/bin/xcodebuild Program ideviceinstaller found in /opt/homebrew/bin/ideviceinstaller ```

Flutter Doctor output

Flutter Doctor output ``` Doctor summary (to see all details, run flutter doctor -v): [✓] Flutter (Channel stable, 3.10.3, on macOS 12.6.1 21G217 darwin-arm64, locale en) [✓] Android toolchain - develop for Android devices (Android SDK version 33.0.1) [✓] Xcode - develop for iOS and macOS (Xcode 14.2) [✓] Chrome - develop for the web [✓] Android Studio (version 2021.3) [✓] VS Code (version 1.79.2) [✓] Connected device (3 available) [✓] Network resources • No issues found!v ```
bartekpacia commented 1 year ago

Hi @eroshinn, thanks for reporting this and sorry for the problem.

I tried running patrol test on iOS Simualtor (iPhone 11, iOS 13.7) and it failed as well, but with a different error. This is something we need to look at.

bartekpacia commented 1 year ago

Here's my output. I have started iPhone 11 simulator running iOS 13.7. It crashed.

Logs ``` $ patrol test -t integration_test/permissions_many_test.dart --device 'iPhone 11' -v Verbose mode enabled. More logs will be printed. Received 1 test target(s) Received test target: /Users/bartek/dev/leancode/patrol/packages/patrol/example/integration_test/permissions_many_test.dart Generated entrypoint /Users/bartek/dev/leancode/patrol/packages/patrol/example/integration_test/test_bundle.dart with 1 bundled test(s) $ flutter --no-version-check devices --machine Received 1 device(s) to run on Received device: iPhone 11 Received 7 --dart-define(s) (0 custom, 7 internal) Received internal --dart-define: PATROL_WAIT=0 Received internal --dart-define: PATROL_APP_PACKAGE_NAME=pl.leancode.patrol.example Received internal --dart-define: PATROL_APP_BUNDLE_ID=pl.leancode.patrol.Example Received internal --dart-define: PATROL_ANDROID_APP_NAME=Patrol example Received internal --dart-define: PATROL_IOS_APP_NAME=Patrol example Received internal --dart-define: INTEGRATION_TEST_SHOULD_REPORT_RESULTS_TO_NATIVE=false Received internal --dart-define: PATROL_TEST_LABEL_ENABLED=true • Building app with entrypoint test_bundle.dart for iOS simulator (debug)... $ flutter build ios --no-version-check --config-only --no-codesign --debug --simulator --target /Users/bartek/dev/leancode/patrol/packages/patrol/example/integration_test/test_bundle.dart --dart-define PATROL_WAIT=0 --dart-define PATROL_APP_PACKAGE_NAME=pl.leancode.patrol.example --dart-define PATROL_APP_BUNDLE_ID=pl.leancode.patrol.Example --dart-define PATROL_ANDROID_APP_NAME=Patrol example --dart-define PATROL_IOS_APP_NAME=Patrol example --dart-define INTEGRATION_TEST_SHOULD_REPORT_RESULTS_TO_NATIVE=false --dart-define PATROL_TEST_LABEL_ENABLED=true Building pl.leancode.patrol.Example for simulator (ios)... $ xcodebuild build-for-testing -workspace Runner.xcworkspace -scheme Runner -configuration Debug -only-testing RunnerUITests -sdk iphonesimulator -destination generic/platform=iOS Simulator -quiet -derivedDataPath ../build/ios_integ OTHER_SWIFT_FLAGS=$(inherited) -D PATROL_ENABLED 2023-07-20 13:43:42.562 xcodebuild[20015:2879898] DVTCoreDeviceEnabledState: DVTCoreDeviceEnabledState_Disabled set via user default (DVTEnableCoreDenote: Building targets in dependency order warning: Run script build phase 'xcode_backend build' will be run during every build because it does not specify any outputs. To address this warning, either add output dependencies to the script phase, or configure it to run in every build by unchecking "Based on dependency analysis" in the script phase. (in target 'Runner' from project 'Runner') warning: Run script build phase 'xcode_backend embed_and_thin' will be run during every build because it does not specify any outputs. To address this warning, either add output dependencies to the script phase, or configure it to run in every build by unchecking "Based on dependency analysis" in the script phase. (in target 'Runner' from project 'Runner') warning: Run script build phase 'xcode_backend build' will be run during every build because it does not specify any outputs. To address this warning, either add output dependencies to the script phase, or configure it to run in every build by unchecking "Based on dependency analysis" in the script phase. (in target 'RunnerUITests' from project 'Runner') warning: Run script build phase 'xcode_backend embed_and_thin' will be run during every build because it does not specify any outputs. To address this warning, either add output dependencies to the script phase, or configure it to run in every build by unchecking "Based on dependency analysis" in the script phase. (in target 'RunnerUITests' from project 'Runner') ✓ Completed building app with entrypoint test_bundle.dart for iOS simulator (44.0s) Will uninstall apps before running tests $ xcrun simctl uninstall 5339525B-2F92-446D-9D3D-8D6E1E1D3A7A pl.leancode.patrol.Example $ xcrun simctl uninstall 5339525B-2F92-446D-9D3D-8D6E1E1D3A7A pl.leancode.patrol.Example.RunnerUITests.xctrunner • Running app with entrypoint test_bundle.dart for iOS simulator on simulator iPhone 11... $ xcodebuild -showsdks -json Assuming SDK version 16.4 for iphonesimulator Looking for .xctestrun matching Runner_iphonesimulator16.4*.xctestrun at /Users/bartek/dev/leancode/patrol/packages/patrol/example/build/ios_integ/Build/Products Found 1 match(es), the first one will be used Found /Users/bartek/dev/leancode/patrol/packages/patrol/example/build/ios_integ/Build/Products/Runner_iphonesimulator16.4-arm64-x86_64.xctestrun $ xcodebuild test-without-building -xctestrun /Users/bartek/dev/leancode/patrol/packages/patrol/example/build/ios_integ/Build/Products/Runner_iphonesimulator16.4-arm64-x86_64.xctestrun -only-testing RunnerUITests -destination platform=iOS Simulator,name=iPhone 11 2023-07-20 13:44:12.278 xcodebuild[22014:2886006] DVTCoreDeviceEnabledState: DVTCoreDeviceEnabledState_Disabled set via user default (DVTEnableCoreDevice=disabled) Command line invocation: /Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild test-without-building -xctestrun /Users/bartek/dev/leancode/patrol/packages/patrol/example/build/ios_integ/Build/Products/Runner_iphonesimulator16.4-arm64-x86_64.xctestrun -only-testing RunnerUITests -destination "platform=iOS Simulator,name=iPhone 11" User defaults from command line: IDEPackageSupportUseBuiltinSCM = YES --- xcodebuild: WARNING: Using the first of multiple matching destinations: { platform:macOS, arch:arm64e, id:00008103-000C258C0EBB001E } { platform:macOS, arch:arm64, id:00008103-000C258C0EBB001E } { platform:macOS, arch:x86_64, id:00008103-000C258C0EBB001E } { platform:macOS, arch:arm64e, variant:Mac Catalyst, id:00008103-000C258C0EBB001E } { platform:macOS, arch:arm64, variant:Mac Catalyst, id:00008103-000C258C0EBB001E } { platform:macOS, arch:x86_64, variant:Mac Catalyst, id:00008103-000C258C0EBB001E } { platform:macOS, arch:arm64e, variant:DriverKit, id:00008103-000C258C0EBB001E } { platform:macOS, arch:arm64, variant:DriverKit, id:00008103-000C258C0EBB001E } { platform:macOS, arch:arm64, variant:Designed for [iPad,iPhone], id:00008103-000C258C0EBB001E } { platform:iOS, id:dvtdevice-DVTiPhonePlaceholder-iphoneos:placeholder, name:Any iOS Device } { platform:iOS Simulator, id:dvtdevice-DVTiOSDeviceSimulatorPlaceholder-iphonesimulator:placeholder, name:Any iOS Simulator Device } { platform:macOS, name:Any Mac } { platform:macOS, variant:Mac Catalyst, name:Any Mac } { platform:tvOS, id:dvtdevice-DVTiOSDevicePlaceholder-appletvos:placeholder, name:Any tvOS Device } { platform:tvOS Simulator, id:dvtdevice-DVTiOSDeviceSimulatorPlaceholder-appletvsimulator:placeholder, name:Any tvOS Simulator Device } { platform:watchOS, id:dvtdevice-DVTiOSDevicePlaceholder-watchos:placeholder, name:Any watchOS Device } { platform:watchOS Simulator, id:dvtdevice-DVTiOSDeviceSimulatorPlaceholder-watchsimulator:placeholder, name:Any watchOS Simulator Device } { platform:iOS Simulator, id:D1F8AF15-E1B7-4E1F-A843-641D902097B2, OS:13.7, name:iPad (7th generation) } { platform:iOS Simulator, id:F1791C9A-4234-4E99-A8A7-2E2DE8DDE8B7, OS:16.4, name:iPad (10th generation) } { platform:iOS Simulator, id:F1791C9A-4234-4E99-A8A7-2E2DE8DDE8B7, OS:16.4, name:iPad (10th generation) } { platform:iOS Simulator, id:A983991C-9FE1-4131-82BD-191C5D596875, OS:13.7, name:iPad Air (3rd generation) } { platform:iOS Simulator, id:91CF5112-DC48-4012-894A-220D87E42E4B, OS:16.4, name:iPad Air (5th generation) } { platform:iOS Simulator, id:91CF5112-DC48-4012-894A-220D87E42E4B, OS:16.4, name:iPad Air (5th generation) } { platform:iOS Simulator, id:F3AE92E3-8D82-443F-A68A-AFE7EF5B4C08, OS:13.7, name:iPad Pro (9.7-inch) } { platform:iOS Simulator, id:9C40035E-A012-4BF0-ADA7-B412C118F7C1, OS:13.7, name:iPad Pro (11-inch) (2nd generation) } { platform:iOS Simulator, id:08F44E9F-0D6B-4724-82DA-5C37227AB201, OS:16.4, name:iPad Pro (11-inch) (4th generation) } { platform:iOS Simulator, id:08F44E9F-0D6B-4724-82DA-5C37227AB201, OS:16.4, name:iPad Pro (11-inch) (4th generation) } { platform:iOS Simulator, id:82B2CA3F-4EB5-47F1-B9AB-921D0AB33EF3, OS:13.7, name:iPad Pro (12.9-inch) (4th generation) } { platform:iOS Simulator, id:FDF5CE16-9B59-47E8-8949-0ABAFDE24540, OS:16.4, name:iPad Pro (12.9-inch) (6th generation) } { platform:iOS Simulator, id:FDF5CE16-9B59-47E8-8949-0ABAFDE24540, OS:16.4, name:iPad Pro (12.9-inch) (6th generation) } { platform:iOS Simulator, id:8524FE2F-FEC2-4FA8-91DA-5C0E65CB29C7, OS:16.4, name:iPad mini (6th generation) } { platform:iOS Simulator, id:8524FE2F-FEC2-4FA8-91DA-5C0E65CB29C7, OS:16.4, name:iPad mini (6th generation) } { platform:iOS Simulator, id:C5E9C9A9-9139-4E09-9E83-6F308F5CB3B0, OS:13.7, name:iPhone 6s } { platform:iOS Simulator, id:5339525B-2F92-446D-9D3D-8D6E1E1D3A7A, OS:13.7, name:iPhone 11 } { platform:iOS Simulator, id:796A67FD-2DF7-474C-A482-BDDA28986058, OS:13.7, name:iPhone 11 Pro } { platform:iOS Simulator, id:D36502FC-E8A8-45D7-A223-3AC3F367D171, OS:13.7, name:iPhone 11 Pro Max } { platform:iOS Simulator, id:2CE2A079-7CC9-4625-9A16-EA439A6BEEF4, OS:16.4, name:iPhone 14 } { platform:iOS Simulator, id:2CE2A079-7CC9-4625-9A16-EA439A6BEEF4, OS:16.4, name:iPhone 14 } { platform:iOS Simulator, id:74FC06CB-8DD0-47F1-883B-FAB120239706, OS:16.4, name:iPhone 14 Plus } { platform:iOS Simulator, id:74FC06CB-8DD0-47F1-883B-FAB120239706, OS:16.4, name:iPhone 14 Plus } { platform:iOS Simulator, id:80F20938-C634-4225-9F4B-6823007057C2, OS:16.4, name:iPhone 14 Pro } { platform:iOS Simulator, id:80F20938-C634-4225-9F4B-6823007057C2, OS:16.4, name:iPhone 14 Pro } { platform:iOS Simulator, id:30801237-660E-4DE2-B37F-92C094B51319, OS:16.4, name:iPhone 14 Pro Max } { platform:iOS Simulator, id:30801237-660E-4DE2-B37F-92C094B51319, OS:16.4, name:iPhone 14 Pro Max } { platform:iOS Simulator, id:E366E3A2-5862-46CB-ACFD-C7A961C03FA0, OS:13.7, name:iPhone SE (2nd generation) } { platform:iOS Simulator, id:849B9EFB-4B11-45ED-9B10-A8F9DFD35EF7, OS:16.4, name:iPhone SE (3rd generation) } { platform:iOS Simulator, id:849B9EFB-4B11-45ED-9B10-A8F9DFD35EF7, OS:16.4, name:iPhone SE (3rd generation) } *** If you believe this error represents a bug, please attach the result bundle at /Users/bartek/Library/Developer/Xcode/DerivedData/Runner-dgfwrkqvtnkwowdqoiwzarzipfcp/Logs/Test/Test-Runner-2023.07.20_1-44-42-+0200.xcresult 2023-07-20 13:44:51.601 xcodebuild[22014:2886006] [MT] IDETestOperationsObserverDebug: 8.749 elapsed -- Testing started completed. 2023-07-20 13:44:51.601 xcodebuild[22014:2886006] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start 2023-07-20 13:44:51.601 xcodebuild[22014:2886006] [MT] IDETestOperationsObserverDebug: 8.749 sec, +8.749 sec -- end Test session results, code coverage, and logs: /Users/bartek/Library/Developer/Xcode/DerivedData/Runner-dgfwrkqvtnkwowdqoiwzarzipfcp/Logs/Test/Test-Runner-2023.07.20_1-44-42-+0200.xcresult Testing failed: Could not launch “RunnerUITests” RunnerUITests-Runner encountered an error (Failed to install or launch the test runner. If you believe this error represents a bug, please attach the result bundle at /Users/bartek/Library/Developer/Xcode/DerivedData/Runner-dgfwrkqvtnkwowdqoiwzarzipfcp/Logs/Test/Test-Runner-2023.07.20_1-44-42-+0200.xcresult. (Underlying Error: Could not launch “RunnerUITests”. The LaunchServices launcher has returned an error. Please check the system logs for the underlying cause of the error. (Underlying Error: The operation couldn’t be completed. (OSStatus error -10661.)))) ** TEST EXECUTE FAILED ** Testing started ✗ Failed to execute tests of app with entrypoint test_bundle.dart for iOS simulator on simulator iPhone 11 (xcodebuild exited with code 65) (44.3s) Error: xcodebuild exited with code 65 #0 throwToolExit (package:patrol_cli/src/base/exceptions.dart:7:3) #1 IOSTestBackend.execute. (package:patrol_cli/src/ios/ios_test_backend.dart:181:9) #2 DisposeScope.run (package:dispose_scope/src/dispose_scope.dart:44:7) #3 IOSTestBackend.execute (package:patrol_cli/src/ios/ios_test_backend.dart:147:5) #4 TestCommand._execute (package:patrol_cli/src/commands/test.dart:269:7) #5 TestCommand.run (package:patrol_cli/src/commands/test.dart:169:23) #6 CommandRunner.runCommand (package:args/command_runner.dart:212:13) #7 PatrolCommandRunner.runCommand (package:patrol_cli/src/runner/patrol_command_runner.dart:310:18) #8 PatrolCommandRunner.run (package:patrol_cli/src/runner/patrol_command_runner.dart:254:18) #9 patrolCommandRunner (package:patrol_cli/src/runner/patrol_command_runner.dart:72:20) #10 main (file:///Users/bartek/dev/leancode/patrol/packages/patrol_cli/bin/main.dart:6:20) See the logs above to learn what happened. Also consider running with --verbose. If the logs still aren't useful, then it's a bug - please report it. $ xcrun simctl uninstall 5339525B-2F92-446D-9D3D-8D6E1E1D3A7A pl.leancode.patrol.Example $ xcrun simctl uninstall 5339525B-2F92-446D-9D3D-8D6E1E1D3A7A pl.leancode.patrol.Example.RunnerUITests.xctrunner ``` 
$ patrol --version
patrol_cli v2.0.4

I also tried running on iPhone 13 simulator running iOS 15.5. It crashed as well.

Logs ``` $ patrol test -t integration_test/example_test.dart -v Verbose mode enabled. More logs will be printed. Received 1 test target(s) Received test target: /Users/bartek/dev/leancode/patrol/packages/patrol/example/integration_test/example_test.dart Generated entrypoint /Users/bartek/dev/leancode/patrol/packages/patrol/example/integration_test/test_bundle.dart with 1 bundled test(s) $ flutter --no-version-check devices --machine No device specified, using the first one (iPhone 13) Received 1 device(s) to run on Received device: iPhone 13 Received 7 --dart-define(s) (0 custom, 7 internal) Received internal --dart-define: PATROL_WAIT=0 Received internal --dart-define: PATROL_APP_PACKAGE_NAME=pl.leancode.patrol.example Received internal --dart-define: PATROL_APP_BUNDLE_ID=pl.leancode.patrol.Example Received internal --dart-define: PATROL_ANDROID_APP_NAME=Patrol example Received internal --dart-define: PATROL_IOS_APP_NAME=Patrol example Received internal --dart-define: INTEGRATION_TEST_SHOULD_REPORT_RESULTS_TO_NATIVE=false Received internal --dart-define: PATROL_TEST_LABEL_ENABLED=true • Building app with entrypoint test_bundle.dart for iOS simulator (debug)... $ flutter build ios --no-version-check --config-only --no-codesign --debug --simulator --target /Users/bartek/dev/leancode/patrol/packages/patrol/example/integration_test/test_bundle.dart --dart-define PATROL_WAIT=0 --dart-define PATROL_APP_PACKAGE_NAME=pl.leancode.patrol.example --dart-define PATROL_APP_BUNDLE_ID=pl.leancode.patrol.Example --dart-define PATROL_ANDROID_APP_NAME=Patrol example --dart-define PATROL_IOS_APP_NAME=Patrol example --dart-define INTEGRATION_TEST_SHOULD_REPORT_RESULTS_TO_NATIVE=false --dart-define PATROL_TEST_LABEL_ENABLED=true Building pl.leancode.patrol.Example for simulator (ios)... $ xcodebuild build-for-testing -workspace Runner.xcworkspace -scheme Runner -configuration Debug -only-testing RunnerUITests -sdk iphonesimulator -destination generic/platform=iOS Simulator -quiet -derivedDataPath ../build/ios_integ OTHER_SWIFT_FLAGS=$(inherited) -D PATROL_ENABLED 2023-07-20 16:41:15.571 xcodebuild[94915:3352120] DVTCoreDeviceEnabledState: DVTCoreDeviceEnabledState_Disabled set via user default (DVTEnableCoreDevice=disabled) /Users/bartek/dev/leancode/patrol/packages/patrol/ios/Classes/AutomatorServer/AutomatorServer.swift:10:17: warning: stored property 'automator' of 'Sendable'-conforming class 'AutomatorServer' has non-sendable type 'Automator' private let automator: Automator ^ /Users/bartek/dev/leancode/patrol/packages/patrol/ios/Classes/AutomatorServer/Automator.swift:4:9: note: class 'Automator' does not conform to the 'Sendable' protocol class Automator { ^ /Users/bartek/dev/leancode/patrol/packages/patrol/ios/Classes/AutomatorServer/AutomatorServer.swift:12:17: warning: stored property 'onAppReady' of 'Sendable'-conforming class 'AutomatorServer' has non-sendable type '(Bool) -> Void' private let onAppReady: (Bool) -> Void ^ /Users/bartek/dev/leancode/patrol/packages/patrol/ios/Classes/AutomatorServer/AutomatorServer.swift:12:17: note: a function type must be marked '@Sendable' to conform to 'Sendable' private let onAppReady: (Bool) -> Void ^ /Users/bartek/dev/leancode/patrol/packages/patrol/ios/Classes/AutomatorServer/AutomatorServer.swift:10:17: warning: stored property 'automator' of 'Sendable'-conforming class 'AutomatorServer' has non-sendable type 'Automator' private let automator: Automator ^ /Users/bartek/dev/leancode/patrol/packages/patrol/ios/Classes/AutomatorServer/Automator.swift:4:9: note: class 'Automator' does not conform to the 'Sendable' protocol class Automator { ^ /Users/bartek/dev/leancode/patrol/packages/patrol/ios/Classes/AutomatorServer/AutomatorServer.swift:12:17: warning: stored property 'onAppReady' of 'Sendable'-conforming class 'AutomatorServer' has non-sendable type '(Bool) -> Void' private let onAppReady: (Bool) -> Void ^ /Users/bartek/dev/leancode/patrol/packages/patrol/ios/Classes/AutomatorServer/AutomatorServer.swift:12:17: note: a function type must be marked '@Sendable' to conform to 'Sendable' private let onAppReady: (Bool) -> Void ^ /Users/bartek/dev/leancode/patrol/packages/patrol/ios/Classes/AutomatorServer/Automator.swift:93:11: warning: variable 'data' was never mutated; consider changing to 'let' constant var data = "\(data)\n" ~~~ ^ let /Users/bartek/dev/leancode/patrol/packages/patrol/ios/Classes/AutomatorServer/Automator.swift:118:11: warning: variable 'data' was never mutated; consider changing to 'let' constant var data = "\(data)\n" ~~~ ^ let /Users/bartek/dev/leancode/patrol/packages/patrol/ios/Classes/AutomatorServer/Automator.swift:659:7: warning: code after 'throw' will never be executed try await runAction(log) { ^ /Users/bartek/dev/leancode/patrol/packages/patrol/ios/Classes/AutomatorServer/AutomatorServer.swift:10:17: warning: stored property 'automator' of 'Sendable'-conforming class 'AutomatorServer' has non-sendable type 'Automator' private let automator: Automator ^ /Users/bartek/dev/leancode/patrol/packages/patrol/ios/Classes/AutomatorServer/Automator.swift:4:9: note: class 'Automator' does not conform to the 'Sendable' protocol class Automator { ^ /Users/bartek/dev/leancode/patrol/packages/patrol/ios/Classes/AutomatorServer/AutomatorServer.swift:12:17: warning: stored property 'onAppReady' of 'Sendable'-conforming class 'AutomatorServer' has non-sendable type '(Bool) -> Void' private let onAppReady: (Bool) -> Void ^ /Users/bartek/dev/leancode/patrol/packages/patrol/ios/Classes/AutomatorServer/AutomatorServer.swift:12:17: note: a function type must be marked '@Sendable' to conform to 'Sendable' private let onAppReady: (Bool) -> Void ^ /Users/bartek/dev/leancode/patrol/packages/patrol/ios/Classes/AutomatorServer/Automator.swift:93:11: warning: variable 'data' was never mutated; consider changing to 'let' constant var data = "\(data)\n" ~~~ ^ let /Users/bartek/dev/leancode/patrol/packages/patrol/ios/Classes/AutomatorServer/Automator.swift:118:11: warning: variable 'data' was never mutated; consider changing to 'let' constant var data = "\(data)\n" ~~~ ^ let /Users/bartek/dev/leancode/patrol/packages/patrol/ios/Classes/AutomatorServer/Automator.swift:659:7: warning: code after 'throw' will never be executed try await runAction(log) { ^ /Users/bartek/dev/leancode/patrol/packages/patrol/ios/Classes/AutomatorServer/AutomatorServer.swift:10:17: warning: stored property 'automator' of 'Sendable'-conforming class 'AutomatorServer' has non-sendable type 'Automator' private let automator: Automator ^ /Users/bartek/dev/leancode/patrol/packages/patrol/ios/Classes/AutomatorServer/Automator.swift:4:9: note: class 'Automator' does not conform to the 'Sendable' protocol class Automator { ^ /Users/bartek/dev/leancode/patrol/packages/patrol/ios/Classes/AutomatorServer/AutomatorServer.swift:12:17: warning: stored property 'onAppReady' of 'Sendable'-conforming class 'AutomatorServer' has non-sendable type '(Bool) -> Void' private let onAppReady: (Bool) -> Void ^ /Users/bartek/dev/leancode/patrol/packages/patrol/ios/Classes/AutomatorServer/AutomatorServer.swift:12:17: note: a function type must be marked '@Sendable' to conform to 'Sendable' private let onAppReady: (Bool) -> Void ^ note: Building targets in dependency order warning: Run script build phase 'xcode_backend build' will be run during every build because it does not specify any outputs. To address this warning, either add output dependencies to the script phase, or configure it to run in every build by unchecking "Based on dependency analysis" in the script phase. (in target 'Runner' from project 'Runner') warning: Run script build phase 'xcode_backend embed_and_thin' will be run during every build because it does not specify any outputs. To address this warning, either add output dependencies to the script phase, or configure it to run in every build by unchecking "Based on dependency analysis" in the script phase. (in target 'Runner' from project 'Runner') warning: Run script build phase 'xcode_backend build' will be run during every build because it does not specify any outputs. To address this warning, either add output dependencies to the script phase, or configure it to run in every build by unchecking "Based on dependency analysis" in the script phase. (in target 'RunnerUITests' from project 'Runner') warning: Run script build phase 'xcode_backend embed_and_thin' will be run during every build because it does not specify any outputs. To address this warning, either add output dependencies to the script phase, or configure it to run in every build by unchecking "Based on dependency analysis" in the script phase. (in target 'RunnerUITests' from project 'Runner') ✓ Completed building app with entrypoint test_bundle.dart for iOS simulator (54.1s) Will uninstall apps before running tests $ xcrun simctl uninstall 18D9FACD-3FCB-4031-9A59-83976C02A88F pl.leancode.patrol.Example $ xcrun simctl uninstall 18D9FACD-3FCB-4031-9A59-83976C02A88F pl.leancode.patrol.Example.RunnerUITests.xctrunner • Running app with entrypoint test_bundle.dart for iOS simulator on simulator iPhone 13... $ xcodebuild -showsdks -json Assuming SDK version 16.4 for iphonesimulator Looking for .xctestrun matching Runner_iphonesimulator16.4*.xctestrun at /Users/bartek/dev/leancode/patrol/packages/patrol/example/build/ios_integ/Build/Products Found 1 match(es), the first one will be used Found /Users/bartek/dev/leancode/patrol/packages/patrol/example/build/ios_integ/Build/Products/Runner_iphonesimulator16.4-arm64-x86_64.xctestrun $ xcodebuild test-without-building -xctestrun /Users/bartek/dev/leancode/patrol/packages/patrol/example/build/ios_integ/Build/Products/Runner_iphonesimulator16.4-arm64-x86_64.xctestrun -only-testing RunnerUITests -destination platform=iOS Simulator,name=iPhone 13 2023-07-20 16:41:52.493 xcodebuild[96856:3358706] DVTCoreDeviceEnabledState: DVTCoreDeviceEnabledState_Disabled set via user default (DVTEnableCoreDevice=disabled) Command line invocation: /Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild test-without-building -xctestrun /Users/bartek/dev/leancode/patrol/packages/patrol/example/build/ios_integ/Build/Products/Runner_iphonesimulator16.4-arm64-x86_64.xctestrun -only-testing RunnerUITests -destination "platform=iOS Simulator,name=iPhone 13" User defaults from command line: IDEPackageSupportUseBuiltinSCM = YES --- xcodebuild: WARNING: Using the first of multiple matching destinations: { platform:macOS, arch:arm64e, id:00008103-000C258C0EBB001E } { platform:macOS, arch:arm64, id:00008103-000C258C0EBB001E } { platform:macOS, arch:x86_64, id:00008103-000C258C0EBB001E } { platform:macOS, arch:arm64e, variant:Mac Catalyst, id:00008103-000C258C0EBB001E } { platform:macOS, arch:arm64, variant:Mac Catalyst, id:00008103-000C258C0EBB001E } { platform:macOS, arch:x86_64, variant:Mac Catalyst, id:00008103-000C258C0EBB001E } { platform:macOS, arch:arm64e, variant:DriverKit, id:00008103-000C258C0EBB001E } { platform:macOS, arch:arm64, variant:DriverKit, id:00008103-000C258C0EBB001E } { platform:macOS, arch:arm64, variant:Designed for [iPad,iPhone], id:00008103-000C258C0EBB001E } { platform:iOS, id:dvtdevice-DVTiPhonePlaceholder-iphoneos:placeholder, name:Any iOS Device } { platform:iOS Simulator, id:dvtdevice-DVTiOSDeviceSimulatorPlaceholder-iphonesimulator:placeholder, name:Any iOS Simulator Device } { platform:macOS, name:Any Mac } { platform:macOS, variant:Mac Catalyst, name:Any Mac } { platform:tvOS, id:dvtdevice-DVTiOSDevicePlaceholder-appletvos:placeholder, name:Any tvOS Device } { platform:tvOS Simulator, id:dvtdevice-DVTiOSDeviceSimulatorPlaceholder-appletvsimulator:placeholder, name:Any tvOS Simulator Device } { platform:watchOS, id:dvtdevice-DVTiOSDevicePlaceholder-watchos:placeholder, name:Any watchOS Device } { platform:watchOS Simulator, id:dvtdevice-DVTiOSDeviceSimulatorPlaceholder-watchsimulator:placeholder, name:Any watchOS Simulator Device } { platform:iOS Simulator, id:D1F8AF15-E1B7-4E1F-A843-641D902097B2, OS:13.7, name:iPad (7th generation) } { platform:iOS Simulator, id:8D22D67C-9BD3-41B1-BA18-3282E5A8C196, OS:15.5, name:iPad (9th generation) } { platform:iOS Simulator, id:8D22D67C-9BD3-41B1-BA18-3282E5A8C196, OS:15.5, name:iPad (9th generation) } { platform:iOS Simulator, id:F1791C9A-4234-4E99-A8A7-2E2DE8DDE8B7, OS:16.4, name:iPad (10th generation) } { platform:iOS Simulator, id:F1791C9A-4234-4E99-A8A7-2E2DE8DDE8B7, OS:16.4, name:iPad (10th generation) } { platform:iOS Simulator, id:A983991C-9FE1-4131-82BD-191C5D596875, OS:13.7, name:iPad Air (3rd generation) } { platform:iOS Simulator, id:760EFD52-4DA0-4B4D-8A0A-5368FCF7E227, OS:15.5, name:iPad Air (5th generation) } { platform:iOS Simulator, id:760EFD52-4DA0-4B4D-8A0A-5368FCF7E227, OS:15.5, name:iPad Air (5th generation) } { platform:iOS Simulator, id:91CF5112-DC48-4012-894A-220D87E42E4B, OS:16.4, name:iPad Air (5th generation) } { platform:iOS Simulator, id:91CF5112-DC48-4012-894A-220D87E42E4B, OS:16.4, name:iPad Air (5th generation) } { platform:iOS Simulator, id:F3AE92E3-8D82-443F-A68A-AFE7EF5B4C08, OS:13.7, name:iPad Pro (9.7-inch) } { platform:iOS Simulator, id:702F0530-3089-4704-8812-AF1B9CBE933E, OS:15.5, name:iPad Pro (9.7-inch) } { platform:iOS Simulator, id:702F0530-3089-4704-8812-AF1B9CBE933E, OS:15.5, name:iPad Pro (9.7-inch) } { platform:iOS Simulator, id:9C40035E-A012-4BF0-ADA7-B412C118F7C1, OS:13.7, name:iPad Pro (11-inch) (2nd generation) } { platform:iOS Simulator, id:7AA3E70A-6BCE-49E8-844D-DFA92E8BFD1A, OS:15.5, name:iPad Pro (11-inch) (3rd generation) } { platform:iOS Simulator, id:7AA3E70A-6BCE-49E8-844D-DFA92E8BFD1A, OS:15.5, name:iPad Pro (11-inch) (3rd generation) } { platform:iOS Simulator, id:08F44E9F-0D6B-4724-82DA-5C37227AB201, OS:16.4, name:iPad Pro (11-inch) (4th generation) } { platform:iOS Simulator, id:08F44E9F-0D6B-4724-82DA-5C37227AB201, OS:16.4, name:iPad Pro (11-inch) (4th generation) } { platform:iOS Simulator, id:82B2CA3F-4EB5-47F1-B9AB-921D0AB33EF3, OS:13.7, name:iPad Pro (12.9-inch) (4th generation) } { platform:iOS Simulator, id:407C2525-C74A-4F33-89DF-04216E4D53E1, OS:15.5, name:iPad Pro (12.9-inch) (5th generation) } { platform:iOS Simulator, id:407C2525-C74A-4F33-89DF-04216E4D53E1, OS:15.5, name:iPad Pro (12.9-inch) (5th generation) } { platform:iOS Simulator, id:FDF5CE16-9B59-47E8-8949-0ABAFDE24540, OS:16.4, name:iPad Pro (12.9-inch) (6th generation) } { platform:iOS Simulator, id:FDF5CE16-9B59-47E8-8949-0ABAFDE24540, OS:16.4, name:iPad Pro (12.9-inch) (6th generation) } { platform:iOS Simulator, id:111B5095-0C37-43A8-A2D3-AFFD4F7CB364, OS:15.5, name:iPad mini (6th generation) } { platform:iOS Simulator, id:111B5095-0C37-43A8-A2D3-AFFD4F7CB364, OS:15.5, name:iPad mini (6th generation) } { platform:iOS Simulator, id:8524FE2F-FEC2-4FA8-91DA-5C0E65CB29C7, OS:16.4, name:iPad mini (6th generation) } { platform:iOS Simulator, id:8524FE2F-FEC2-4FA8-91DA-5C0E65CB29C7, OS:16.4, name:iPad mini (6th generation) } { platform:iOS Simulator, id:C5E9C9A9-9139-4E09-9E83-6F308F5CB3B0, OS:13.7, name:iPhone 6s } { platform:iOS Simulator, id:0774BCBA-3982-445F-9DA2-81219902EF2E, OS:15.5, name:iPhone 8 } { platform:iOS Simulator, id:0774BCBA-3982-445F-9DA2-81219902EF2E, OS:15.5, name:iPhone 8 } { platform:iOS Simulator, id:09CA3C30-56A2-4748-B455-64B21C8E8CBB, OS:15.5, name:iPhone 8 Plus } { platform:iOS Simulator, id:09CA3C30-56A2-4748-B455-64B21C8E8CBB, OS:15.5, name:iPhone 8 Plus } { platform:iOS Simulator, id:5339525B-2F92-446D-9D3D-8D6E1E1D3A7A, OS:13.7, name:iPhone 11 } { platform:iOS Simulator, id:802B39C2-1813-46CD-92EE-A5B560CFB348, OS:15.5, name:iPhone 11 } { platform:iOS Simulator, id:802B39C2-1813-46CD-92EE-A5B560CFB348, OS:15.5, name:iPhone 11 } { platform:iOS Simulator, id:796A67FD-2DF7-474C-A482-BDDA28986058, OS:13.7, name:iPhone 11 Pro } { platform:iOS Simulator, id:0AAF4355-FF62-40FB-B186-369901A0568F, OS:15.5, name:iPhone 11 Pro } { platform:iOS Simulator, id:0AAF4355-FF62-40FB-B186-369901A0568F, OS:15.5, name:iPhone 11 Pro } { platform:iOS Simulator, id:D36502FC-E8A8-45D7-A223-3AC3F367D171, OS:13.7, name:iPhone 11 Pro Max } { platform:iOS Simulator, id:12B86CF2-9614-43F2-8FC1-3A568214AA0B, OS:15.5, name:iPhone 11 Pro Max } { platform:iOS Simulator, id:12B86CF2-9614-43F2-8FC1-3A568214AA0B, OS:15.5, name:iPhone 11 Pro Max } { platform:iOS Simulator, id:35BA6AA6-07EB-4F1A-A5F5-05E4BF9B19FB, OS:15.5, name:iPhone 12 } { platform:iOS Simulator, id:35BA6AA6-07EB-4F1A-A5F5-05E4BF9B19FB, OS:15.5, name:iPhone 12 } { platform:iOS Simulator, id:F36BBD32-124E-400D-8056-CC1775F478D8, OS:15.5, name:iPhone 12 Pro } { platform:iOS Simulator, id:F36BBD32-124E-400D-8056-CC1775F478D8, OS:15.5, name:iPhone 12 Pro } { platform:iOS Simulator, id:61720D00-1A67-44DE-B681-2264FE41DE9E, OS:15.5, name:iPhone 12 Pro Max } { platform:iOS Simulator, id:61720D00-1A67-44DE-B681-2264FE41DE9E, OS:15.5, name:iPhone 12 Pro Max } { platform:iOS Simulator, id:ABEC3DBC-3880-4C06-A509-D511B9EF8F19, OS:15.5, name:iPhone 12 mini } { platform:iOS Simulator, id:ABEC3DBC-3880-4C06-A509-D511B9EF8F19, OS:15.5, name:iPhone 12 mini } { platform:iOS Simulator, id:18D9FACD-3FCB-4031-9A59-83976C02A88F, OS:15.5, name:iPhone 13 } { platform:iOS Simulator, id:18D9FACD-3FCB-4031-9A59-83976C02A88F, OS:15.5, name:iPhone 13 } { platform:iOS Simulator, id:83689950-1FEA-4201-B23A-8C355619386C, OS:15.5, name:iPhone 13 Pro } { platform:iOS Simulator, id:83689950-1FEA-4201-B23A-8C355619386C, OS:15.5, name:iPhone 13 Pro } { platform:iOS Simulator, id:9464ADDC-E688-444A-B5F0-65FA4D0AB74E, OS:15.5, name:iPhone 13 Pro Max } { platform:iOS Simulator, id:9464ADDC-E688-444A-B5F0-65FA4D0AB74E, OS:15.5, name:iPhone 13 Pro Max } { platform:iOS Simulator, id:DD2EE67E-1BB6-41D6-B895-DDE2D9C4F41E, OS:15.5, name:iPhone 13 mini } { platform:iOS Simulator, id:DD2EE67E-1BB6-41D6-B895-DDE2D9C4F41E, OS:15.5, name:iPhone 13 mini } { platform:iOS Simulator, id:2CE2A079-7CC9-4625-9A16-EA439A6BEEF4, OS:16.4, name:iPhone 14 } { platform:iOS Simulator, id:2CE2A079-7CC9-4625-9A16-EA439A6BEEF4, OS:16.4, name:iPhone 14 } { platform:iOS Simulator, id:74FC06CB-8DD0-47F1-883B-FAB120239706, OS:16.4, name:iPhone 14 Plus } { platform:iOS Simulator, id:74FC06CB-8DD0-47F1-883B-FAB120239706, OS:16.4, name:iPhone 14 Plus } { platform:iOS Simulator, id:80F20938-C634-4225-9F4B-6823007057C2, OS:16.4, name:iPhone 14 Pro } { platform:iOS Simulator, id:80F20938-C634-4225-9F4B-6823007057C2, OS:16.4, name:iPhone 14 Pro } { platform:iOS Simulator, id:30801237-660E-4DE2-B37F-92C094B51319, OS:16.4, name:iPhone 14 Pro Max } { platform:iOS Simulator, id:30801237-660E-4DE2-B37F-92C094B51319, OS:16.4, name:iPhone 14 Pro Max } { platform:iOS Simulator, id:E366E3A2-5862-46CB-ACFD-C7A961C03FA0, OS:13.7, name:iPhone SE (2nd generation) } { platform:iOS Simulator, id:420F5FEC-4A91-4C40-A70B-6A8F424902A8, OS:15.5, name:iPhone SE (3rd generation) } { platform:iOS Simulator, id:420F5FEC-4A91-4C40-A70B-6A8F424902A8, OS:15.5, name:iPhone SE (3rd generation) } { platform:iOS Simulator, id:849B9EFB-4B11-45ED-9B10-A8F9DFD35EF7, OS:16.4, name:iPhone SE (3rd generation) } { platform:iOS Simulator, id:849B9EFB-4B11-45ED-9B10-A8F9DFD35EF7, OS:16.4, name:iPhone SE (3rd generation) } { platform:iOS Simulator, id:D89B7C09-A85D-42D1-8612-DB02CB93600C, OS:15.5, name:iPod touch (7th generation) } { platform:iOS Simulator, id:D89B7C09-A85D-42D1-8612-DB02CB93600C, OS:15.5, name:iPod touch (7th generation) } *** If you believe this error represents a bug, please attach the result bundle at /Users/bartek/Library/Developer/Xcode/DerivedData/Runner-gywmfcdjtlnkxechrhffbzwtmbhd/Logs/Test/Test-Runner-2023.07.20_4-42-22-+0200.xcresult 2023-07-20 16:42:32.596 xcodebuild[96856:3358706] [MT] IDETestOperationsObserverDebug: 9.330 elapsed -- Testing started completed. 2023-07-20 16:42:32.596 xcodebuild[96856:3358706] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start 2023-07-20 16:42:32.596 xcodebuild[96856:3358706] [MT] IDETestOperationsObserverDebug: 9.330 sec, +9.330 sec -- end Test session results, code coverage, and logs: /Users/bartek/Library/Developer/Xcode/DerivedData/Runner-gywmfcdjtlnkxechrhffbzwtmbhd/Logs/Test/Test-Runner-2023.07.20_4-42-22-+0200.xcresult Testing failed: Could not launch “RunnerUITests” RunnerUITests-Runner encountered an error (Failed to install or launch the test runner. If you believe this error represents a bug, please attach the result bundle at /Users/bartek/Library/Developer/Xcode/DerivedData/Runner-gywmfcdjtlnkxechrhffbzwtmbhd/Logs/Test/Test-Runner-2023.07.20_4-42-22-+0200.xcresult. (Underlying Error: Could not launch “RunnerUITests”. The LaunchServices launcher has returned an error. Please check the system logs for the underlying cause of the error. (Underlying Error: The operation couldn’t be completed. (OSStatus error -10661.)))) ** TEST EXECUTE FAILED ** Testing started ✗ Failed to execute tests of app with entrypoint test_bundle.dart for iOS simulator on simulator iPhone 13 (xcodebuild exited with code 65) (44.7s) Error: xcodebuild exited with code 65 #0 throwToolExit (package:patrol_cli/src/base/exceptions.dart:7:3) #1 IOSTestBackend.execute. (package:patrol_cli/src/ios/ios_test_backend.dart:181:9) #2 DisposeScope.run (package:dispose_scope/src/dispose_scope.dart:44:7) #3 IOSTestBackend.execute (package:patrol_cli/src/ios/ios_test_backend.dart:147:5) #4 TestCommand._execute (package:patrol_cli/src/commands/test.dart:269:7) #5 TestCommand.run (package:patrol_cli/src/commands/test.dart:169:23) #6 CommandRunner.runCommand (package:args/command_runner.dart:212:13) #7 PatrolCommandRunner.runCommand (package:patrol_cli/src/runner/patrol_command_runner.dart:310:18) #8 PatrolCommandRunner.run (package:patrol_cli/src/runner/patrol_command_runner.dart:254:18) #9 patrolCommandRunner (package:patrol_cli/src/runner/patrol_command_runner.dart:72:20) #10 main (file:///Users/bartek/dev/leancode/patrol/packages/patrol_cli/bin/main.dart:6:20) See the logs above to learn what happened. Also consider running with --verbose. If the logs still aren't useful, then it's a bug - please report it. $ xcrun simctl uninstall 18D9FACD-3FCB-4031-9A59-83976C02A88F pl.leancode.patrol.Example $ xcrun simctl uninstall 18D9FACD-3FCB-4031-9A59-83976C02A88F pl.leancode.patrol.Example.RunnerUITests.xctrunner ```
bartekpacia commented 1 year ago

Firebase Test Lab also dropped support for iOS 13. I'm going to run any other non-16 version.

bartekpacia commented 1 year ago

I managed to run on iPhone 13 simulator running iOS 15.5.

First run patrol build ios --debug --simulator.

Then go to the ios directory and run:

xcodebuild test-without-building \
  -xctestrun /Users/bartek/dev/leancode/patrol/packages/patrol/example/build/ios_integ/Build/Products/Runner_iphonesimulator16.4-arm64-x86_64.xctestrun \
  -only-testing RunnerUITests \
  -destination 'platform=iOS Simulator,name=iPhone 13,OS=15.5'

The only difference between the above xcodebuild invocation and the one that patrol_cli does internally is in the -destination:

Unfortunately, for some reason, running with -destination 'platform=iOS Simulator,name=iPhone 11,OS=13.7' crashes right after the initial run. Unknown cause.

fylyppo commented 9 months ago

Related to https://github.com/leancodepl/patrol/discussions/1662