WebDriver/Appium cannot see non-cell views added to UICollectionView

joe-mcg commented 7 years ago

The problem

Appium is apparently unable to see non-cell views added to UICollectionViews in iOS. We've tried this with a couple of different applications now and have seen the same thing.

Accessibility IDs within these views can be seen by the Accessibility Inspector, and by XCUITest.

Environment

Appium version: 1.6.5
Last Appium version that did not exhibit the issue (if applicable): Not known.
Desktop OS/version used to run Appium: OS X 10.12.3
Node.js version (unless using Appium.app|exe): node --version v7.10.0. npm --version 4.2.0
Mobile platform/version under test: iOS 10.3/10.2
Real device or emulator/simulator: Simulator
Appium CLI or Appium.app|exe: CLI

Link to Appium logs

Create a GIST which is a paste of your full Appium logs, and link them here. Do NOT paste your full Appium logs here, as it will make this issue very long and hard to read! If you are reporting a bug, always include Appium logs!

Code To Reproduce Issue [ Good To Have ]

Here is a sample project demonstrating the issue: https://github.com/joe-mcg/AppiumTest

Appium only sees the following:

<?xml version=“1.0” encoding=“UTF-8"?><AppiumAUT><XCUIElementTypeApplication type=“XCUIElementTypeApplication” name=“AppiumTest” label=“AppiumTest” enabled=“true” visible=“true” x=“0" y=“0” width=“375" height=“667”> <XCUIElementTypeWindow type=“XCUIElementTypeWindow” enabled=“true” visible=“true” x=“0” y=“0" width=“375” height=“667"> <XCUIElementTypeOther type=“XCUIElementTypeOther” enabled=“true” visible=“true” x=“0" y=“0” width=“375" height=“667”> <XCUIElementTypeCollectionView type=“XCUIElementTypeCollectionView” enabled=“true” visible=“true” x=“0” y=“0" width=“375” height=“667"> <XCUIElementTypeCell type=“XCUIElementTypeCell” enabled=“true” visible=“true” x=“0" y=“375” width=“50" height=“50”> <XCUIElementTypeOther type=“XCUIElementTypeOther” enabled=“true” visible=“true” x=“0” y=“375" width=“50” height=“50"/> </XCUIElementTypeCell> <XCUIElementTypeCell type=“XCUIElementTypeCell” enabled=“true” visible=“true” x=“54" y=“375” width=“50" height=“50”> <XCUIElementTypeOther type=“XCUIElementTypeOther” enabled=“true” visible=“true” x=“54” y=“375" width=“50” height=“50"/> </XCUIElementTypeCell> <XCUIElementTypeCell type=“XCUIElementTypeCell” enabled=“true” visible=“true” x=“108" y=“375” width=“51" height=“50”> <XCUIElementTypeOther type=“XCUIElementTypeOther” enabled=“true” visible=“true” x=“108” y=“375" width=“51” height=“50"/> </XCUIElementTypeCell> <XCUIElementTypeCell type=“XCUIElementTypeCell” enabled=“true” visible=“true” x=“162" y=“375” width=“51" height=“50”> <XCUIElementTypeOther type=“XCUIElementTypeOther” enabled=“true” visible=“true” x=“162” y=“375" width=“51” height=“50"/> </XCUIElementTypeCell> <XCUIElementTypeCell type=“XCUIElementTypeCell” enabled=“true” visible=“true” x=“216" y=“375” width=“51" height=“50”> <XCUIElementTypeOther type=“XCUIElementTypeOther” enabled=“true” visible=“true” x=“216” y=“375" width=“51” height=“50"/> </XCUIElementTypeCell> </XCUIElementTypeCollectionView> </XCUIElementTypeOther> </XCUIElementTypeWindow> <XCUIElementTypeWindow type=“XCUIElementTypeWindow” enabled=“true” visible=“false” x=“0" y=“0” width=“375" height=“667”> <XCUIElementTypeOther type=“XCUIElementTypeOther” enabled=“true” visible=“false” x=“0” y=“0" width=“375” height=“667"/> </XCUIElementTypeWindow> <XCUIElementTypeWindow type=“XCUIElementTypeWindow” enabled=“true” visible=“true” x=“0" y=“0” width=“375" height=“667”> <XCUIElementTypeStatusBar type=“XCUIElementTypeStatusBar” enabled=“true” visible=“true” x=“0” y=“0" width=“375” height=“20"> <XCUIElementTypeOther type=“XCUIElementTypeOther” enabled=“true” visible=“false” x=“0" y=“0” width=“375" height=“20”/> <XCUIElementTypeOther type=“XCUIElementTypeOther” enabled=“true” visible=“true” x=“0” y=“0" width=“375” height=“20"> <XCUIElementTypeOther type=“XCUIElementTypeOther” enabled=“true” visible=“true” x=“6" y=“0” width=“39" height=“20”/> <XCUIElementTypeOther type=“XCUIElementTypeOther” value=“SSID” name=“3 of 3 Wi-Fi bars” label=“3 of 3 Wi-Fi bars” enabled=“true” visible=“true” x=“50" y=“0” width=“13" height=“20”/> <XCUIElementTypeOther type=“XCUIElementTypeOther” name=“10:26 AM” label=“10:26 AM” enabled=“true” visible=“true” x=“161” y=“0" width=“56” height=“20"/> <XCUIElementTypeOther type=“XCUIElementTypeOther” name=“-100% battery power” label=“-100% battery power” enabled=“true” visible=“true” x=“337" y=“0” width=“33" height=“20”/> </XCUIElementTypeOther> </XCUIElementTypeStatusBar> </XCUIElementTypeWindow> </XCUIElementTypeApplication></AppiumAUT>

Whereas XCUITest sees the following: (lldb) po print(XCUIApplication().debugDescription) t = 25.86s Snapshot accessibility hierarchy for com.tomirving.AppiumTest Attributes: Application 0x17016f9c0: {{0.0, 0.0}, {375.0, 667.0}}, label: 'AppiumTest' Element subtree: →Application 0x17016f9c0: {{0.0, 0.0}, {375.0, 667.0}}, label: 'AppiumTest' Window 0x170172840: Main Window, {{0.0, 0.0}, {375.0, 667.0}} Other 0x1701711c0: traits: 8589934592, {{0.0, 0.0}, {375.0, 667.0}} CollectionView 0x170170d40: traits: 35192962023424, {{0.0, 0.0}, {375.0, 667.0}} Other 0x1701717c0: traits: 8589934592, {{0.0, 0.0}, {375.0, 375.0}} Other 0x170171040: traits: 8589934592, {{10.0, 10.0}, {355.0, 355.0}}, identifier: 'Background Identifier', label: 'Background Label' Button 0x170171580: traits: 8589934593, {{147.5, 147.5}, {80.0, 80.0}}, identifier: 'Button Identifier', label: 'Button Label' Cell 0x170171400: traits: 8589934592, {{0.0, 375.0}, {50.0, 50.0}} Other 0x1701714c0: traits: 8589934592, {{0.0, 375.0}, {50.0, 50.0}} Cell 0x170170c80: traits: 8589934592, {{54.0, 375.0}, {50.0, 50.0}} Other 0x170170bc0: traits: 8589934592, {{54.0, 375.0}, {50.0, 50.0}} Cell 0x170170b00: traits: 8589934592, {{108.5, 375.0}, {50.0, 50.0}} Other 0x170170e00: traits: 8589934592, {{108.5, 375.0}, {50.0, 50.0}} Cell 0x170170a40: traits: 8589934592, {{162.5, 375.0}, {50.0, 50.0}} Other 0x1701708c0: traits: 8589934592, {{162.5, 375.0}, {50.0, 50.0}} Cell 0x170170980: traits: 8589934592, {{216.5, 375.0}, {50.0, 50.0}} Other 0x170170800: traits: 8589934592, {{216.5, 375.0}, {50.0, 50.0}} Window 0x170171100: {{0.0, 0.0}, {375.0, 667.0}} StatusBar 0x170170740: {{0.0, 0.0}, {375.0, 20.0}} Other 0x170170680: {{0.0, 0.0}, {375.0, 20.0}} Other 0x170171a00: {{0.0, 0.0}, {375.0, 20.0}} Other 0x170171c40: traits: 8388608, {{6.0, 0.0}, {35.0, 20.0}}, label: '3 of 5 bars, signal strength' Other 0x170171b80: traits: 8388608, {{44.0, 0.0}, {73.0, 20.0}}, label: 'vodafone UK network' Other 0x170171dc0: traits: 8388608, {{122.0, 0.0}, {13.0, 20.0}}, label: '3 of 3 Wi-Fi bars', value: SSID Other 0x170171d00: traits: 8389120, {{173.0, 0.0}, {32.0, 20.0}}, label: '12:13' Other 0x170171f40: traits: 8388608, {{309.0, 0.0}, {61.0, 20.0}}, label: '95% battery power, Charging' Other 0x170171e80: traits: 8388608, {{309.0, 0.0}, {25.0, 20.0}}, label: '95% battery power' Path to element: →Application 0x17016f9c0: {{0.0, 0.0}, {375.0, 667.0}}, label: 'AppiumTest' Query chain: →Find: Target Application 0x1700b12e0 Output: { Application 0x17016f9c0: {{0.0, 0.0}, {375.0, 667.0}}, label: 'AppiumTest' }

travstatham commented 7 years ago

I'm not sure if the issue I'm getting is related, but Appium-desktop inspector on an ios app shows cells as visible = false even though I can visibly see them in the inspector itself. There are elements in the tree hierarchy aligned to each cell, and you can even see information like label or enabled, but the visibility is basically opposite of reality.

marekcirkos commented 7 years ago

@joe-mcg Do you have accessibility annotations on that views? @dem0n0cracy Inspector will show all elements reported by accessibility server, regardless of visibility. You can check visibility property on elements details in inspector as well.

HasaanAli commented 7 years ago

I'm also having this issue.

joe-mcg commented 7 years ago

Hi Marek, we are definitely using accessibility annotations.

On Mon, 10 Jul 2017 at 11:17, Marek Cirkos notifications@github.com wrote:

@joe-mcg https://github.com/joe-mcg Do you have accessibility annotations on that views? @dem0n0cracy https://github.com/dem0n0cracy Inspector will show all elements reported by accessibility server, regardless of visibility. You can check visibility property on elements details in inspector as well.

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/facebook/WebDriverAgent/issues/625#issuecomment-314065350, or mute the thread https://github.com/notifications/unsubscribe-auth/ABFqOl5KNVCJZYtDSWP21Hf8uByoBGmLks5sMfo7gaJpZM4N-E8K .

marekcirkos commented 7 years ago

I am sorry, but unless you give me some concrete example to reproduce it. I cannot work on this.

joe-mcg commented 7 years ago

Marek, in the initial bug I raised I attached a sample project which explicitly shows the issue. I'm not sure what you mean about not having an example?

marekcirkos commented 7 years ago

Oh, I missed it. Sorry for that. Will have a look when I got some time. Thanks!

MansiVirmani commented 6 years ago

Any updates on this? I have been facing a similar problem like this where I have a list of contacts, and when I scroll down and try to inspect that cell, it points to the first element of the list and not the respective one. Its visibility is set as false when its already visible on the screen. I am stuck on this and any help would be great. Thanks.

facebookarchive / WebDriverAgent