search

mapbox / ios-sdk-examples

Live example code for the Mapbox iOS SDK
https://www.mapbox.com/ios-sdk/examples/
Other
95 stars 35 forks source link

Tests can fail with kAXErrorServerNotFound #358

Open friedbunny opened 4 years ago

friedbunny commented 4 years ago

Tests on CI can spuriously fail with:

ExamplesUITests
    ✗ testEveryExample, Failed to get matching snapshot: Error getting main window kAXErrorServerNotFound
ExamplesUITests
  testEveryExample, Failed to get matching snapshot: Error getting main window kAXErrorServerNotFound
  ExamplesUITests.m:168
     Executed 1 test, with 1 failure (0 unexpected) in 270.239 (270.242) seconds

Saw this in two recent PRs: #354 and #356.

friedbunny commented 4 years ago

This seems to manifest in a few different ways, but always with the UI test application dying:

https://app.bitrise.io/build/2d5a7b9eb9084967 from #352

t =   110.00s Add a line style layer from GeoJSON (Swift)
    t =   110.02s     Tap Cell
    t =   110.02s         Wait for com.mapbox.examples to idle
    t =   110.05s         Find the Cell
    t =   110.05s             Requesting snapshot of accessibility hierarchy for app with pid 2829
    t =   110.18s             Find: Descendants matching type Table
    t =   110.18s             Find: Descendants matching type Cell
    t =   110.18s             Find: Element at index 15
    t =   110.19s         Check for interrupting elements affecting Cell
    t =   110.20s         Synthesize event
    t =   110.23s             Scroll element to visible
    t =   110.23s             Find the Cell
    t =   110.24s                 Requesting snapshot of accessibility hierarchy for app with pid 2829
    t =   110.36s                 Find: Descendants matching type Table
    t =   110.36s                 Find: Descendants matching type Cell
    t =   110.36s                 Find: Element at index 15
    t =   110.43s         Wait for com.mapbox.examples to idle
    t =   110.50s     Get all elements bound by index for: Descendants matching type Button
    t =   110.61s         Assertion Failure: ExamplesUITests.m:173: Failed to resolve query: Lost connection to the application (pid 2829).
    t =   110.62s Tear Down
Test Case '-[ExamplesUITests testEveryExample]' failed (110.619 seconds).
Test Suite 'ExamplesUITests' failed at 2019-09-20 16:58:03.350.
     Executed 1 test, with 1 failure (0 unexpected) in 110.619 (110.620) seconds
friedbunny commented 4 years ago

Not really sure yet what the cause is here, but I suspect it’s Xcode 11 funkiness.

I’m also intrigued by the possibly of moving to individualized tests of examples (#333), rather than the current monolithic testEveryExample — it’s hard to know if one long XCUITest is more or less reliable than many short ones.

srafiqa commented 4 years ago

I am facing the same issue when running from CI system, when running with XCODE 11 and fastlane Error as : , Failed to get matching snapshots: Error getting main window kAXErrorServerNotFound also adb crashed in

markanderson4 commented 4 years ago

I'm seeing this error as well, running XCode 11.1

eb-smith-affirm commented 4 years ago

Me too. I'm running Xcode 12.2.1. It seems to be an Apple bug with no work around known but I'd love a solution.

oliver98844 commented 4 years ago

Same with Xcode 11.2.1

iosdeveloperIN commented 4 years ago

I am facing same issue on Xcode 11.2.1.

v-i-s-h-a-l commented 4 years ago

Me too. I'm running Xcode 12.2.1. It seems to be an Apple bug with no work around known but I'd love a solution.

Woah, are you from future :P

VasilijSviridov commented 4 years ago

same here xCode 11.2.1 happens 1 in 5 runs :/

VasilijSviridov commented 4 years ago

and basically happens when you using wait for element function, but if you'll give that function some kind of Thread sleep for at least 10 seconds, you'll be alright, not the best workaround but works for me :/

eb-smith-affirm commented 4 years ago

@v-i-s-h-a-l I am from the future but don't tell anybody.

Seems a little better in Xcode 11.3.1 but sure it's fixed yet.

satheeshkumarf commented 4 years ago

Not sure, why we still get this error "Failed to get matching snapshots:", even though the XCUITest documentation says it uses remote queries and there is no snapshot at all… Any insights on this?

captainbarbosa commented 3 years ago

Might be seeing this again with Xcode 12 in https://github.com/mapbox/ios-sdk-examples/pull/411#issuecomment-692339227.

GintsRomanovskis commented 1 year ago

Is there any fixes available?