[v1.32.0] Tests occasionally fail with "Unknown error" (MacOS 13.5.2 & iOS 17)

[RevillWeb]

Describe the bug Quite occasionally some of my tests fail with "Unknown Error", it appears to be completely random and I have no reproduction steps other than if I run my full regression (which includes 21 tests) one of two of the tests will fail with "Unknown Error". Looking at the maestro logs it appears to be related to clearing the keychain.

If a test fails in this way, the next test will start immediately and work correctly, all my tests clear the keychain at the start.

If someone could direct me to where I could fetch more logs, I'd be happy to do so. Any help would be greatly appreciated, I'm thoroughly enjoying using Maestro and it is proving valuable but having tests fail this way so regularly could get in the way of that.

[INFO ] m.cli.runner.TestSuiteInteractor - Run ../../../scripts/release-roles.js RUNNING
[INFO ] m.cli.runner.TestSuiteInteractor - Run ../../../scripts/release-roles.js COMPLETED
[INFO ] m.cli.runner.TestSuiteInteractor - Apply configuration RUNNING
[INFO ] m.cli.runner.TestSuiteInteractor - Apply configuration COMPLETED
[INFO ] m.cli.runner.TestSuiteInteractor - Run ../../../navigation/launch.yaml RUNNING
[INFO ] m.cli.runner.TestSuiteInteractor - Apply configuration RUNNING
[INFO ] m.cli.runner.TestSuiteInteractor - Apply configuration COMPLETED
[INFO ] m.cli.runner.TestSuiteInteractor - Clear keychain RUNNING
[INFO ] maestro.Maestro - Clearing keychain
[INFO ] util.CommandLineUtils - Running command line operation: [xcrun, simctl, spawn, AFC4E88C-E2A5-4A34-8D19-997C776054E0, launchctl, stop, com.apple.securityd]
[INFO ] util.CommandLineUtils - Running command line operation: [rm, -rf, /Users/maestro/Library/Developer/CoreSimulator/Devices/AFC4E88C-E2A5-4A34-8D19-997C776054E0/data/Library/Keychains]
[ERROR] util.CommandLineUtils - Process failed with exit code 1
[ERROR] util.CommandLineUtils - 
[INFO ] m.cli.runner.TestSuiteInteractor - Clear keychain FAILED
[INFO ] maestro.Maestro - Taking screenshot: /var/folders/51/h8xk7ps14rgcfxdhndr21s0h0000gn/T/screenshot-16951707807483001099945022172524.png
[INFO ] maestro.utils.ScreenshotUtils - Taking screenshot to output sink
[INFO ] m.cli.runner.TestSuiteInteractor - Run ../../../navigation/launch.yaml FAILED

To Reproduce Unknown

Expected behavior The keychain is correctly cleared and the test continues.

Environment information (please complete the following information):

• Maestro version v1.32.0
• Platform: iOS 17
• Framework: Native iOS
• Device model and OS version: iPhone SE (3rd Generation) iOS 17
• Simulator
• Host: Macbook Mini M2

Workspace (if applicable) Please share your workspace with Flow files and application binary if you are able to do so as that greatly helps in reproduction efforts.

Bugreport output (if applicable) Doesn't seem to show anything useful but I've attached it anyway.

[RevillWeb]

If I update to maestro 1.33.0 all my tests consistently fail with "Unable to clear state for app X", maybe that could be a clue?

[JemmaLangford321]

I am having same issue as above. Are there any work arounds for this?

[DavidREntwistle]

I have noticed this same issue on 1.37.4 also.

[DavidREntwistle]

Update, this still appears to be an issue in CLI 1.37.9