[v1.30.2] Failed to reach XCUITest Server in restart

[tqirazevedo]

Describe the bug When I run the maestro test command, the following error occurs:

maestro test ios-advanced-flow.yaml

Running on iPhone 14 Pro Max - iOS 16.2 - 308A7B68-C45F-4D9B-AE1E-BD952577DC9A

xcuitest.XCTestDriverClient$XCTestDriverUnreachable: Failed to reach XCUITest Server in restart at xcuitest.XCTestDriverClient.restartXCTestRunnerService(XCTestDriverClient.kt:47) at ios.xctest.XCTestIOSDevice.open(XCTestIOSDevice.kt:31) at ios.LocalIOSDevice.open(LocalIOSDevice.kt:27) at maestro.drivers.IOSDriver.open(IOSDriver.kt:82) at maestro.Maestro$Companion.ios(Maestro.kt:579) at maestro.cli.session.MaestroSessionManager.createIOS(MaestroSessionManager.kt:329) at maestro.cli.session.MaestroSessionManager.createMaestro(MaestroSessionManager.kt:157) at maestro.cli.session.MaestroSessionManager.access$createMaestro(MaestroSessionManager.kt:50) at maestro.cli.session.MaestroSessionManager$newSession$session$1.invoke(MaestroSessionManager.kt:82) at maestro.cli.session.MaestroSessionManager$newSession$session$1.invoke(MaestroSessionManager.kt:81) at maestro.cli.db.KeyValueStore.withExclusiveLock(KeyValueStore.kt:37) at maestro.cli.session.SessionStore.withExclusiveLock(SessionStore.kt:74) at maestro.cli.session.MaestroSessionManager.newSession(MaestroSessionManager.kt:81) at maestro.cli.session.MaestroSessionManager.newSession$default(MaestroSessionManager.kt:58) at maestro.cli.command.TestCommand.call(TestCommand.kt:126) at maestro.cli.command.TestCommand.call(TestCommand.kt:44) at picocli.CommandLine.executeUserObject(CommandLine.java:1933) at picocli.CommandLine.access$1200(CommandLine.java:145) at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2332) at picocli.CommandLine$RunLast.handle(CommandLine.java:2326) at picocli.CommandLine$RunLast.handle(CommandLine.java:2291) at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2159) at maestro.cli.DisableAnsiMixin$Companion.executionStrategy(DisableAnsiMixin.kt:22) at picocli.CommandLine.execute(CommandLine.java:2058) at maestro.cli.AppKt.main(App.kt:131)

To Reproduce

1. Start the simulator
2. Run maestro test ios-advanced-flow.yaml
3. The error occurs after executing the maestro test command. No flow steps are executed.

Expected behavior Install the application and run the tests.

Screenshots

Environment information (please complete the following information):

• Maestro version [v1.30.2]
• Platform: [macOS Monterey 12.6.3 (21G419)]
• Framework: [Native iOS]
• Device model and OS version: [Iphome 14 Pro Max iOS 16.2]
• Simulator
• Host [MacBook Pro (16-inch, 2019)]

Workspace (if applicable) Maestro version: 1.30.2 OS: Mac OS X OS version: 12.6.3 Architecture: x86_64 Processor: 2,3 GHz Intel Core i9 8-Core

Bugreport output xctest_runner_logs.zip 2023-07-17_134123.zip

Additional context

[amanjeetsingh150]

Hey @tqirazevedo it seems like the underneath xctest failed. Was this working before or this is first time?

Can you also share xcresult in this directory which would have logs on why test failed:

Test session results, code coverage, and logs:
    /var/folders/b0/wmy7sptn3g7fybky7wx9p915c3klrz/T/maestro_xctestrunner_xcodebuild_output11490501882164145638/Logs/Test/Test-maestro-driver-ios-2023.07.17_13-41-46--0300.xcresult

Testing started

[tqirazevedo]

Hi, @amanjeetsingh150 follows logs as requested. Arquivo Comprimido.zip

[amanjeetsingh150]

Hey @tqirazevedo can you help me understand the environment for debugging:

• Is this the first time on this system you are trying maestro? Has it worked before?
• Are you using multiple simulators or just one
• Is this for the CI system or locally?
• Is this M1 or Intel mac?

[tqirazevedo]

Hi, @amanjeetsingh150 Follow the required informations

• Is this the first time on this system that you are trying to maestro? It's the first time
• Has it worked before? no
• Are you using multiple simulators or just one? only one
• Is this for the CI system or locally? locally
• Is this M1 or Intel mac? intel mac

[KarlosQ]

The same for me since 1.30.0 (I use Bitrise M1)

[denil-ct]

I too get the same issue on 1.30.4 But interestingly I get it only when calling it via a shell script. If I call maestro directly from the terminal it works fine. Also, after the error message pops up in terminal, I can see the maestro runner app get installed three times, for the 3 retries I believe, and then if I call the same command again, it installs again and proceeds normally.

Update: It seems the issue only happens on booting two simulators simultaneously. However I do not execute flows on them in parallel. Only after one flow completes on a device, I switch to another device to execute the next flow.

[solidcell]

I'm running into the same issue on 1.30.4 on CI. Everything works as expected locally on my laptop.

Is this the first time on this system that you are trying to maestro? First time Has it worked before? No Are you using multiple simulators or just one? Only one is booted (verified by first running xcrun simctl list) Is this for the CI system or locally? CircleCI Is this M1 or Intel mac? Resource class: macos.x86.medium.gen2 with executor macos and xcode: 14.3.1

[sregg]

Same here. The last frame of the recorded video of the simulator (i.e. xcrun simctl io booted recordVideo video_record.mov) is the driver app being installed:

Is this the first time on this system that you are trying to maestro? First time Has it worked before? No Are you using multiple simulators or just one? Only one Is this for the CI system or locally? Github Actions Is this M1 or Intel mac? Intel macos-latest

[sregg]

I got more luck setting MAESTRO_DRIVER_STARTUP_TIMEOUT to 60000

[solidcell]

I got more luck setting MAESTRO_DRIVER_STARTUP_TIMEOUT to 60000

That did the trick, thanks! I should have continued reading more of the docs. It took almost 30 seconds on CI, so longer than the default timeout of 15 seconds.

[solidcell]

Actually, that seems to only allow it to work seldomly. Even if I crank it up to 1200000 (20 minutes), the entire CI build only runs for 14 minutes, so I know it's timing out earlier than I'm specifying. There must be some max internally that is still not quite enough for CI.

[solidcell]

Haha, diving into the code and it looks like MAESTRO_DRIVER_STARTUP_TIMEOUT is Android-only. It's only used in AndroidDriver.kt and not in IOSDriver.kt at all. Upon looking at the docs again, it's listed under an Android header, so that makes sense. I wonder how difficult it would be to implement this for iOS. I'll give it a shot.

I guess I got "lucky" with the very next run I had after using MAESTRO_DRIVER_STARTUP_TIMEOUT. Strange coincidence.

[solidcell]

Here's a PR I made which adds iOS support for MAESTRO_DRIVER_STARTUP_TIMEOUT

[solidcell]

Since I'm not sure how long it might take to get that PR merged, here's a workaround I came up with you can use when calling maestro:

i=0
while [ $i -le 9 ]; do
maestro test && break
let i=i+1
done

Realistically you can turn down the number of attempts from 9 to something like 2 or 3, but it doesn't hurt if you're sure it should be launching. Any previous attempt will have already kicked off the process of launching, so it's not like you're rolling the dice and need so many "attempts". The next try will just piggyback off the headstart.

Of course, this will also re-run the command even if it succeeds to launch but then fails because there were actually failing tests. This solution could be modified, but just FYI. I won't bother for now, since I'm hoping the PR gets merged before I need it. Right now, I'm only integrating Maestro and only have a single simple test for integration purposes.

[radhakrishnanakireddy]

facing this issue a lot after upgrading ios 16.6 and could you please review and fix this issue

[iusn]

have the same issues

maestro test test_flow.yaml Running on iPhone 15 Pro - iOS 17.0 - C42FA8FB-3389-4EAB-B3D9-9058228F7238

java.util.concurrent.TimeoutException: Maestro iOS driver did not start up in time at maestro.drivers.IOSDriver.awaitLaunch(IOSDriver.kt:475) at maestro.drivers.IOSDriver.open(IOSDriver.kt:64) at maestro.Maestro$Companion.ios(Maestro.kt:574) at maestro.cli.session.MaestroSessionManager.createIOS(MaestroSessionManager.kt:309) at maestro.cli.session.MaestroSessionManager.createMaestro(MaestroSessionManager.kt:154) at maestro.cli.session.MaestroSessionManager.access$createMaestro(MaestroSessionManager.kt:49) at maestro.cli.session.MaestroSessionManager$newSession$session$1.invoke(MaestroSessionManager.kt:82) at maestro.cli.session.MaestroSessionManager$newSession$session$1.invoke(MaestroSessionManager.kt:81) at maestro.cli.db.KeyValueStore.withExclusiveLock(KeyValueStore.kt:37) at maestro.cli.session.SessionStore.withExclusiveLock(SessionStore.kt:74) at maestro.cli.session.MaestroSessionManager.newSession(MaestroSessionManager.kt:81) at maestro.cli.session.MaestroSessionManager.newSession$default(MaestroSessionManager.kt:58) at maestro.cli.command.TestCommand.call(TestCommand.kt:136) at maestro.cli.command.TestCommand.call(TestCommand.kt:46) at picocli.CommandLine.executeUserObject(CommandLine.java:1933) at picocli.CommandLine.access$1200(CommandLine.java:145) at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2332) at picocli.CommandLine$RunLast.handle(CommandLine.java:2326) at picocli.CommandLine$RunLast.handle(CommandLine.java:2291) at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2159) at maestro.cli.DisableAnsiMixin$Companion.executionStrategy(DisableAnsiMixin.kt:22) at picocli.CommandLine.execute(CommandLine.java:2058) at maestro.cli.AppKt.main(App.kt:117)

maestro 1.33.1 Platform : Mac OS 14.0 Framework: iOS 17.0 Device model and OS version: [Iphome 15 Pro iOS 17.0] Simulator Host: MacBook Pro 14-inch, 2021

[Irandoust]

I did see that the PR for MAESTRO_DRIVER_STARTUP_TIMEOUT in iOS is merged. I Have the same issue in iOS 16.2 in Azure pipelines and increasing MAESTRO_DRIVER_STARTUP_TIMEOUT didn't work. Any ideas?

[Irandoust]

Since I'm not sure how long it might take to get that PR merged, here's a workaround I came up with you can use when calling maestro:

i=0
while [ $i -le 9 ]; do
maestro test && break
let i=i+1
done

Realistically you can turn down the number of attempts from 9 to something like 2 or 3, but it doesn't hurt if you're sure it should be launching. Any previous attempt will have already kicked off the process of launching, so it's not like you're rolling the dice and need so many "attempts". The next try will just piggyback off the headstart.

Of course, this will also re-run the command even if it succeeds to launch but then fails because there were actually failing tests. This solution could be modified, but just FYI. I won't bother for now, since I'm hoping the PR gets merged before I need it. Right now, I'm only integrating Maestro and only have a single simple test for integration purposes.

I added this loop, the first time I got the java.util.concurrent.TimeoutException: Maestro iOS driver did not start up in time error but then I got this:

java.net.SocketException: Protocol family unavailable
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:607)
    at okhttp3.internal.platform.Platform.connectSocket(Platform.kt:128)
    at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.kt:295)
    at okhttp3.internal.connection.RealConnection.connect(RealConnection.kt:207)
    at okhttp3.internal.connection.ExchangeFinder.findConnection(ExchangeFinder.kt:226)
    at okhttp3.internal.connection.ExchangeFinder.findHealthyConnection(ExchangeFinder.kt:106)
    at okhttp3.internal.connection.ExchangeFinder.find(ExchangeFinder.kt:74)
    at okhttp3.internal.connection.RealCall.initExchange$okhttp(RealCall.kt:255)
    at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:32)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
    at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:95)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
    at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:83)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
    at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:76)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
    at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:201)
    at okhttp3.internal.connection.RealCall.execute(RealCall.kt:154)
    at xcuitest.installer.LocalXCTestInstaller.xcTestDriverStatusCheck(LocalXCTestInstaller.kt:131)
    at xcuitest.installer.LocalXCTestInstaller.isChannelAlive(LocalXCTestInstaller.kt:96)
    at xcuitest.XCTestDriverClient.isChannelAlive(XCTestDriverClient.kt:177)
    at ios.xctest.XCTestIOSDevice.isShutdown(XCTestIOSDevice.kt:174)
    at maestro.cli.session.MaestroSessionManager.createIOS(MaestroSessionManager.kt:311)
    at maestro.cli.session.MaestroSessionManager.createMaestro(MaestroSessionManager.kt:154)
    at maestro.cli.session.MaestroSessionManager.access$createMaestro(MaestroSessionManager.kt:49)
    at maestro.cli.session.MaestroSessionManager$newSession$session$1.invoke(MaestroSessionManager.kt:82)
    at maestro.cli.session.MaestroSessionManager$newSession$session$1.invoke(MaestroSessionManager.kt:81)
    at maestro.cli.db.KeyValueStore.withExclusiveLock(KeyValueStore.kt:37)
    at maestro.cli.session.SessionStore.withExclusiveLock(SessionStore.kt:74)
    at maestro.cli.session.MaestroSessionManager.newSession(MaestroSessionManager.kt:81)
    at maestro.cli.session.MaestroSessionManager.newSession$default(MaestroSessionManager.kt:58)
    at maestro.cli.command.TestCommand.call(TestCommand.kt:136)
    at maestro.cli.command.TestCommand.call(TestCommand.kt:46)
    at picocli.CommandLine.executeUserObject(CommandLine.java:1933)
    at picocli.CommandLine.access$1200(CommandLine.java:145)
    at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2332)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2326)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2291)
    at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2159)
    at maestro.cli.DisableAnsiMixin$Companion.executionStrategy(DisableAnsiMixin.kt:22)
    at picocli.CommandLine.execute(CommandLine.java:2058)
    at maestro.cli.AppKt.main(App.kt:117)

Any thoughts?

[grsoares-zoe]

I added this loop, the first time I got the java.util.concurrent.TimeoutException: Maestro iOS driver did not start up in time error but then I got this:

@Irandoust I'm getting the same java.net.SocketException: Protocol family unavailable issue. Have  you managed to find an workaraound for this?

[Freire71]

Any updates on this? I'm facing this locally