kif-framework / KIF

Keep It Functional - An iOS Functional Testing Framework
Other
6.2k stars 912 forks source link

NSInternalInconsistencyException scrolling UITableViews #1007

Closed aeioliu closed 6 years ago

aeioliu commented 6 years ago

Hi there - we're getting this error using the latest version of KIF on any waitForView(withAccessibilityLabel:) or tapView(withAccessibilityLabel:) that tries to interact with a UITableView:

Attempted to dequeue multiple cells for the same index path, which is not allowed. If you really need to dequeue more cells than the table view is requesting, use the -dequeueReusableCellWithIdentifier: method (without an index path). Cell identifier: cell, index path: <NSIndexPath: 0x600000a360a0> {length = 2, path = 0 - 7}

The offending code seems to be in UIView+KIFAdditions.m, line 254:

UITableViewCell *cell = [tableView.dataSource tableView:tableView cellForRowAtIndexPath:indexPath];

It's looking like to us that the issue is related to how that function looping through all the cells in a tableview to find its position, which either fails or takes a considerable amount of time, causing the operation to retry, or the tests to hang/fail. Commenting out this code (lines 252 - 273) allows the tests to progress naturally without interruption. Also, adding a significant amount of time padding (5-10s) for the view to load in with tester().wait(forTimeInterval:) before calling waitForView or tapView seems to help, though this adds significant time to testing.

Even though the code is reused for collectionviews, it doesn't seem like

Seems like this is only an issue with Xcode 9 and 8.3.3 is unaffected. Stack trace:

caught "NSInternalInconsistencyException", "Attempted to dequeue multiple cells for the same index path, which is not allowed. If you really need to dequeue more cells than the table view is requesting, use the -dequeueReusableCellWithIdentifier: method (without an index path). Cell identifier: cell, index path: <NSIndexPath: 0x60c000a3e0e0> {length = 2, path = 0 - 7}"
(
    0   CoreFoundation                      0x000000011374f1cb __exceptionPreprocess + 171
    1   libobjc.A.dylib                     0x00000001184adf41 objc_exception_throw + 48
    2   CoreFoundation                      0x0000000113754362 +[NSException raise:format:arguments:] + 98
    3   Foundation                          0x0000000117f52089 -[NSAssertionHandler handleFailureInMethod:object:file:lineNumber:description:] + 193
    4   UIKit                               0x0000000115b737b4 -[UITableView _dequeueReusableCellWithIdentifier:forIndexPath:usingPresentationValues:] + 454
    5   UIKit                               0x0000000115b735ba -[UITableView dequeueReusableCellWithIdentifier:forIndexPath:] + 89
    6   UIKit                               0x000000013cfdc4b3 -[UITableViewAccessibility dequeueReusableCellWithIdentifier:forIndexPath:] + 285
    7   DoordashConsumer                    0x000000011044b7c3 _T016DoordashConsumer19OrderListDatasourceC9tableViewSo07UITableG4CellCSo0hG0C_10Foundation9IndexPathV12cellForRowAttF + 1907
    8   DoordashConsumer                    0x000000011044ba6c _T016DoordashConsumer19OrderListDatasourceC9tableViewSo07UITableG4CellCSo0hG0C_10Foundation9IndexPathV12cellForRowAttFTo + 92
    9   KIF                                 0x0000000139d2f81e -[UIView(KIFAdditions) accessibilityElementMatchingBlock:notHidden:] + 3822
    10  KIF                                 0x0000000139d2e8f8 -[UIView(KIFAdditions) accessibilityElementMatchingBlock:] + 72
    11  KIF                                 0x0000000139d2ec8e -[UIView(KIFAdditions) accessibilityElementMatchingBlock:notHidden:] + 862
    12  KIF                                 0x0000000139d2e8f8 -[UIView(KIFAdditions) accessibilityElementMatchingBlock:] + 72
    13  KIF                                 0x0000000139d2ec8e -[UIView(KIFAdditions) accessibilityElementMatchingBlock:notHidden:] + 862
    14  KIF                                 0x0000000139d2e8f8 -[UIView(KIFAdditions) accessibilityElementMatchingBlock:] + 72
    15  KIF                                 0x0000000139d2ec8e -[UIView(KIFAdditions) accessibilityElementMatchingBlock:notHidden:] + 862
    16  KIF                                 0x0000000139d2e8f8 -[UIView(KIFAdditions) accessibilityElementMatchingBlock:] + 72
    17  KIF                                 0x0000000139d2ec8e -[UIView(KIFAdditions) accessibilityElementMatchingBlock:notHidden:] + 862
    18  KIF                                 0x0000000139d2e8f8 -[UIView(KIFAdditions) accessibilityElementMatchingBlock:] + 72
    19  KIF                                 0x0000000139d2ec8e -[UIView(KIFAdditions) accessibilityElementMatchingBlock:notHidden:] + 862
    20  KIF                                 0x0000000139d2e8f8 -[UIView(KIFAdditions) accessibilityElementMatchingBlock:] + 72
    21  KIF                                 0x0000000139d2ec8e -[UIView(KIFAdditions) accessibilityElementMatchingBlock:notHidden:] + 862
    22  KIF                                 0x0000000139d2e8f8 -[UIView(KIFAdditions) accessibilityElementMatchingBlock:] + 72
    23  KIF                                 0x0000000139d2ec8e -[UIView(KIFAdditions) accessibilityElementMatchingBlock:notHidden:] + 862
    24  KIF                                 0x0000000139d2e8f8 -[UIView(KIFAdditions) accessibilityElementMatchingBlock:] + 72
    25  KIF                                 0x0000000139d2ec8e -[UIView(KIFAdditions) accessibilityElementMatchingBlock:notHidden:] + 862
    26  KIF                                 0x0000000139d2e8f8 -[UIView(KIFAdditions) accessibilityElementMatchingBlock:] + 72
    27  KIF                                 0x0000000139d2e079 -[UIView(KIFAdditions) accessibilityElementWithLabel:accessibilityValue:traits:] + 249
    28  KIF                                 0x0000000139d2731e -[UIApplication(KIFAdditions) accessibilityElementWithLabel:accessibilityValue:traits:] + 542
    29  KIF                                 0x0000000139d24cb1 +[UIAccessibilityElement(KIFAdditions) accessibilityElementWithLabel:value:traits:fromRootView:error:] + 417
    30  KIF                                 0x0000000139d24222 +[UIAccessibilityElement(KIFAdditions) accessibilityElement:view:withLabel:value:traits:fromRootView:tappable:error:] + 258
    31  KIF                                 0x0000000139d240db +[UIAccessibilityElement(KIFAdditions) accessibilityElement:view:withLabel:value:traits:tappable:error:] + 235
    32  KIF                                 0x0000000139d0e5b2 __83-[KIFUITestActor waitForAccessibilityElement:view:withLabel:value:traits:tappable:]_block_invoke + 114
    33  KIF                                 0x0000000139d0a5b7 -[KIFTestActor tryRunningBlock:complete:timeout:error:] + 167
    34  KIF                                 0x0000000139d0a8a9 -[KIFTestActor runBlock:complete:timeout:] + 137
    35  KIF                                 0x0000000139d0a9b4 -[KIFTestActor runBlock:complete:] + 148
    36  KIF                                 0x0000000139d0aa90 -[KIFTestActor runBlock:] + 64
    37  KIF                                 0x0000000139d0e4db -[KIFUITestActor waitForAccessibilityElement:view:withLabel:value:traits:tappable:] + 331
    38  KIF                                 0x0000000139d0e30e -[KIFUITestActor waitForViewWithAccessibilityLabel:value:traits:tappable:] + 206
    39  KIF                                 0x0000000139d0df60 -[KIFUITestActor waitForViewWithAccessibilityLabel:traits:] + 96
    40  DoordashConsumerFunctionalTests     0x0000000139caaa36 _T031DoordashConsumerFunctionalTests14A2CheckoutSpecC4specyyFyycfU_yycfU6_yycfU0_ + 214
    41  Quick                               0x0000000139f59c96 _T05Quick7ExampleC3runyyF + 2198
    42  Quick                               0x0000000139f5a8c4 _T05Quick7ExampleC3runyyFTo + 36
    43  Quick                               0x0000000139f5139b __60+[QuickSpec addInstanceMethodForExample:classSelectorNames:]_block_invoke + 123
    44  CoreFoundation                      0x00000001136d356c __invoking___ + 140
    45  CoreFoundation                      0x00000001136d3440 -[NSInvocation invoke] + 320
    46  XCTest                              0x00000001387a1949 __24-[XCTestCase invokeTest]_block_invoke + 591
    47  XCTest                              0x00000001387e9f45 -[XCUITestContext performInScope:] + 183
    48  XCTest                              0x00000001387a16ef -[XCTestCase invokeTest] + 141
    49  XCTest                              0x00000001387a26b0 __26-[XCTestCase performTest:]_block_invoke.369 + 42
    50  XCTest                              0x00000001387eec4b +[XCTContext runInContextForTestCase:block:] + 163
    51  XCTest                              0x00000001387a204c -[XCTestCase performTest:] + 608
    52  XCTest                              0x000000013879e052 __27-[XCTestSuite performTest:]_block_invoke + 363
    53  XCTest                              0x000000013879d9b9 -[XCTestSuite _performProtectedSectionForTest:testSection:] + 26
    54  XCTest                              0x000000013879dbb6 -[XCTestSuite performTest:] + 239
    55  XCTest                              0x000000013879e052 __27-[XCTestSuite performTest:]_block_invoke + 363
    56  XCTest                              0x000000013879d9b9 -[XCTestSuite _performProtectedSectionForTest:testSection:] + 26
    57  XCTest                              0x000000013879dbb6 -[XCTestSuite performTest:] + 239
    58  XCTest                              0x000000013879e052 __27-[XCTestSuite performTest:]_block_invoke + 363
    59  XCTest                              0x000000013879d9b9 -[XCTestSuite _performProtectedSectionForTest:testSection:] + 26
    60  XCTest                              0x000000013879dbb6 -[XCTestSuite performTest:] + 239
    61  XCTest                              0x00000001387f616d __44-[XCTTestRunSession runTestsAndReturnError:]_block_invoke + 40
    62  XCTest                              0x00000001387b1232 -[XCTestObservationCenter _observeTestExecutionForBlock:] + 475
    63  XCTest                              0x00000001387f600c -[XCTTestRunSession runTestsAndReturnError:] + 281
    64  XCTest                              0x000000013878d6ab -[XCTestDriver runTestsAndReturnError:] + 314
    65  XCTest                              0x00000001387edeb6 _XCTestMain + 619
    66  CoreFoundation                      0x00000001136f220c __CFRUNLOOP_IS_CALLING_OUT_TO_A_BLOCK__ + 12
    67  CoreFoundation                      0x00000001136d6a3b __CFRunLoopDoBlocks + 203
    68  CoreFoundation                      0x00000001136d6214 __CFRunLoopRun + 1300
    69  CoreFoundation                      0x00000001136d5a89 CFRunLoopRunSpecific + 409
    70  GraphicsServices                    0x000000011d5f79c6 GSEventRunModal + 62
    71  UIKit                               0x0000000115a35d30 UIApplicationMain + 159
    72  DoordashConsumer                    0x000000010f615227 main + 55
    73  libdyld.dylib                       0x0000000119ffcd81 start + 1
    74  ???                                 0x0000000000000005 0x0 + 5
mikelupo commented 6 years ago

This smells like an iOS bug. @Justinmartin @royalpineapple, curious for your ideas here.

Sent from my iPhone

On Sep 18, 2017, at 11:35 PM, Steven Liu notifications@github.com wrote:

Hi there - we're getting this error using the latest version of KIF on any waitForView(withAccessibilityLabel:) or tapView(withAccessibilityLabel:) that tries to interact with a UITableView:

Attempted to dequeue multiple cells for the same index path, which is not allowed. If you really need to dequeue more cells than the table view is requesting, use the -dequeueReusableCellWithIdentifier: method (without an index path). Cell identifier: cell, index path: <NSIndexPath: 0x600000a360a0> {length = 2, path = 0 - 7} The offending code seems to be in UIView+KIFAdditions.m, line 254:

UITableViewCell *cell = [tableView.dataSource tableView:tableView cellForRowAtIndexPath:indexPath];

It's looking like to us that the issue is related to how that function looping through all the cells in a tableview to find its position, which either fails or takes a considerable amount of time, causing the operation to retry, or the tests to hang/fail. Commenting out this code (lines 252 - 273) allows the tests to progress naturally without interruption.

Even though the code is reused for collectionviews, it doesn't seem like

Seems like this is only an issue with Xcode 9 and 8.3.3 is unaffected. Stack trace:

caught "NSInternalInconsistencyException", "Attempted to dequeue multiple cells for the same index path, which is not allowed. If you really need to dequeue more cells than the table view is requesting, use the -dequeueReusableCellWithIdentifier: method (without an index path). Cell identifier: cell, index path: <NSIndexPath: 0x60c000a3e0e0> {length = 2, path = 0 - 7}" ( 0 CoreFoundation 0x000000011374f1cb exceptionPreprocess + 171 1 libobjc.A.dylib 0x00000001184adf41 objc_exception_throw + 48 2 CoreFoundation 0x0000000113754362 +[NSException raise:format:arguments:] + 98 3 Foundation 0x0000000117f52089 -[NSAssertionHandler handleFailureInMethod:object:file:lineNumber:description:] + 193 4 UIKit 0x0000000115b737b4 -[UITableView _dequeueReusableCellWithIdentifier:forIndexPath:usingPresentationValues:] + 454 5 UIKit 0x0000000115b735ba -[UITableView dequeueReusableCellWithIdentifier:forIndexPath:] + 89 6 UIKit 0x000000013cfdc4b3 -[UITableViewAccessibility dequeueReusableCellWithIdentifier:forIndexPath:] + 285 7 DoordashConsumer 0x000000011044b7c3 _T016DoordashConsumer19OrderListDatasourceC9tableViewSo07UITableG4CellCSo0hG0C_10Foundation9IndexPathV12cellForRowAttF + 1907 8 DoordashConsumer 0x000000011044ba6c _T016DoordashConsumer19OrderListDatasourceC9tableViewSo07UITableG4CellCSo0hG0C_10Foundation9IndexPathV12cellForRowAttFTo + 92 9 KIF 0x0000000139d2f81e -[UIView(KIFAdditions) accessibilityElementMatchingBlock:notHidden:] + 3822 10 KIF 0x0000000139d2e8f8 -[UIView(KIFAdditions) accessibilityElementMatchingBlock:] + 72 11 KIF 0x0000000139d2ec8e -[UIView(KIFAdditions) accessibilityElementMatchingBlock:notHidden:] + 862 12 KIF 0x0000000139d2e8f8 -[UIView(KIFAdditions) accessibilityElementMatchingBlock:] + 72 13 KIF 0x0000000139d2ec8e -[UIView(KIFAdditions) accessibilityElementMatchingBlock:notHidden:] + 862 14 KIF 0x0000000139d2e8f8 -[UIView(KIFAdditions) accessibilityElementMatchingBlock:] + 72 15 KIF 0x0000000139d2ec8e -[UIView(KIFAdditions) accessibilityElementMatchingBlock:notHidden:] + 862 16 KIF 0x0000000139d2e8f8 -[UIView(KIFAdditions) accessibilityElementMatchingBlock:] + 72 17 KIF 0x0000000139d2ec8e -[UIView(KIFAdditions) accessibilityElementMatchingBlock:notHidden:] + 862 18 KIF 0x0000000139d2e8f8 -[UIView(KIFAdditions) accessibilityElementMatchingBlock:] + 72 19 KIF 0x0000000139d2ec8e -[UIView(KIFAdditions) accessibilityElementMatchingBlock:notHidden:] + 862 20 KIF 0x0000000139d2e8f8 -[UIView(KIFAdditions) accessibilityElementMatchingBlock:] + 72 21 KIF 0x0000000139d2ec8e -[UIView(KIFAdditions) accessibilityElementMatchingBlock:notHidden:] + 862 22 KIF 0x0000000139d2e8f8 -[UIView(KIFAdditions) accessibilityElementMatchingBlock:] + 72 23 KIF 0x0000000139d2ec8e -[UIView(KIFAdditions) accessibilityElementMatchingBlock:notHidden:] + 862 24 KIF 0x0000000139d2e8f8 -[UIView(KIFAdditions) accessibilityElementMatchingBlock:] + 72 25 KIF 0x0000000139d2ec8e -[UIView(KIFAdditions) accessibilityElementMatchingBlock:notHidden:] + 862 26 KIF 0x0000000139d2e8f8 -[UIView(KIFAdditions) accessibilityElementMatchingBlock:] + 72 27 KIF 0x0000000139d2e079 -[UIView(KIFAdditions) accessibilityElementWithLabel:accessibilityValue:traits:] + 249 28 KIF 0x0000000139d2731e -[UIApplication(KIFAdditions) accessibilityElementWithLabel:accessibilityValue:traits:] + 542 29 KIF 0x0000000139d24cb1 +[UIAccessibilityElement(KIFAdditions) accessibilityElementWithLabel:value:traits:fromRootView:error:] + 417 30 KIF 0x0000000139d24222 +[UIAccessibilityElement(KIFAdditions) accessibilityElement:view:withLabel:value:traits:fromRootView:tappable:error:] + 258 31 KIF 0x0000000139d240db +[UIAccessibilityElement(KIFAdditions) accessibilityElement:view:withLabel:value:traits:tappable:error:] + 235 32 KIF 0x0000000139d0e5b2 83-[KIFUITestActor waitForAccessibilityElement:view:withLabel:value:traits:tappable:]_block_invoke + 114 33 KIF 0x0000000139d0a5b7 -[KIFTestActor tryRunningBlock:complete:timeout:error:] + 167 34 KIF 0x0000000139d0a8a9 -[KIFTestActor runBlock:complete:timeout:] + 137 35 KIF 0x0000000139d0a9b4 -[KIFTestActor runBlock:complete:] + 148 36 KIF 0x0000000139d0aa90 -[KIFTestActor runBlock:] + 64 37 KIF 0x0000000139d0e4db -[KIFUITestActor waitForAccessibilityElement:view:withLabel:value:traits:tappable:] + 331 38 KIF 0x0000000139d0e30e -[KIFUITestActor waitForViewWithAccessibilityLabel:value:traits:tappable:] + 206 39 KIF 0x0000000139d0df60 -[KIFUITestActor waitForViewWithAccessibilityLabel:traits:] + 96 40 DoordashConsumerFunctionalTests 0x0000000139caaa36 _T031DoordashConsumerFunctionalTests14A2CheckoutSpecC4specyyFyycfU_yycfU6yycfU0 + 214 41 Quick 0x0000000139f59c96 _T05Quick7ExampleC3runyyF + 2198 42 Quick 0x0000000139f5a8c4 _T05Quick7ExampleC3runyyFTo + 36 43 Quick 0x0000000139f5139b 60+[QuickSpec addInstanceMethodForExample:classSelectorNames:]_block_invoke + 123 44 CoreFoundation 0x00000001136d356c _invoking + 140 45 CoreFoundation 0x00000001136d3440 -[NSInvocation invoke] + 320 46 XCTest 0x00000001387a1949 24-[XCTestCase invokeTest]_block_invoke + 591 47 XCTest 0x00000001387e9f45 -[XCUITestContext performInScope:] + 183 48 XCTest 0x00000001387a16ef -[XCTestCase invokeTest] + 141 49 XCTest 0x00000001387a26b0 26-[XCTestCase performTest:]_block_invoke.369 + 42 50 XCTest 0x00000001387eec4b +[XCTContext runInContextForTestCase:block:] + 163 51 XCTest 0x00000001387a204c -[XCTestCase performTest:] + 608 52 XCTest 0x000000013879e052 27-[XCTestSuite performTest:]_block_invoke + 363 53 XCTest 0x000000013879d9b9 -[XCTestSuite _performProtectedSectionForTest:testSection:] + 26 54 XCTest 0x000000013879dbb6 -[XCTestSuite performTest:] + 239 55 XCTest 0x000000013879e052 __27-[XCTestSuite performTest:]_block_invoke + 363 56 XCTest 0x000000013879d9b9 -[XCTestSuite _performProtectedSectionForTest:testSection:] + 26 57 XCTest 0x000000013879dbb6 -[XCTestSuite performTest:] + 239 58 XCTest 0x000000013879e052 27-[XCTestSuite performTest:]_block_invoke + 363 59 XCTest 0x000000013879d9b9 -[XCTestSuite _performProtectedSectionForTest:testSection:] + 26 60 XCTest 0x000000013879dbb6 -[XCTestSuite performTest:] + 239 61 XCTest 0x00000001387f616d 44-[XCTTestRunSession runTestsAndReturnError:]_block_invoke + 40 62 XCTest 0x00000001387b1232 -[XCTestObservationCenter _observeTestExecutionForBlock:] + 475 63 XCTest 0x00000001387f600c -[XCTTestRunSession runTestsAndReturnError:] + 281 64 XCTest 0x000000013878d6ab -[XCTestDriver runTestsAndReturnError:] + 314 65 XCTest 0x00000001387edeb6 _XCTestMain + 619 66 CoreFoundation 0x00000001136f220c CFRUNLOOP_IS_CALLING_OUT_TO_A_BLOCK + 12 67 CoreFoundation 0x00000001136d6a3b CFRunLoopDoBlocks + 203 68 CoreFoundation 0x00000001136d6214 __CFRunLoopRun + 1300 69 CoreFoundation 0x00000001136d5a89 CFRunLoopRunSpecific + 409 70 GraphicsServices 0x000000011d5f79c6 GSEventRunModal + 62 71 UIKit 0x0000000115a35d30 UIApplicationMain + 159 72 DoordashConsumer 0x000000010f615227 main + 55 73 libdyld.dylib 0x0000000119ffcd81 start + 1 74 ??? 0x0000000000000005 0x0 + 5 — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

jak0br commented 6 years ago

I'm having the exact same issue. A test suite using a tableview works when I run it stand alone but not when I run it together with other suites. It works when I build it with Xcode 9 and run it in a iOS 10.3.1 simulator but not when I run it in iOS 11 simulator.

justinseanmartin commented 6 years ago

Seems like a new restriction introduced by iOS 11 in the interest of app performance I guess.

Sounds like there is at least a stopgap workaround of not using the iOS 11 sim in the short term, but not something we can live with.

@jak0br @aeioliu - Do either of you know if this happens with the existing KIF test suite? I know there is a test for tableviews. I don't have Xcode 9 installed on my machine, but I guess today is as good as any day to upgrade.

aeioliu commented 6 years ago

Doesn't look like it happens in the tableview tests in the test suite - there is a unrelated failing test testMoveRowDown though in there.

There's a few tests that fail though (for example interacting with a UIAlertView) that we've also noticed (will create a separate issue for it).

smileyborg commented 6 years ago

UIKit engineer here, just to provide some context this is a new assert in iOS 11 that was added to prevent misusing this API on UITableView. The dequeueReusableCellWithIdentifier:forIndexPath: method should only be called once inside of tableView:cellForRowAtIndexPath: in response to a callback from the table view. This method results in side effects (in particular, the cell returned is configured internally in preparation for it to be displayed in the table view) that will cause serious bugs if multiple cells are dequeued for the same index path, so as of iOS 11 this now asserts to prevent these bugs from happening in the first place.

Most clients hit this assert because they mistakenly call their own implementation of tableView:cellForRowAtIndexPath: from somewhere else, instead of the normal case where the table view calls that method, which results in a cell being dequeued for the passed index path that the table view did not ask for. In most cases, the client is doing this to (incorrectly) try to get the visible cell for that index path from the table view, which is correctly done by asking the table view directly using cellForRowAtIndexPath: on the UITableView.

For unit testing like KIF here, if you're trying to iterate over all cells in the table view, you are best off doing that by programmatically scrolling the table view, forcing a layout pass on it so that the visible cells are updated, asking the table view for visible cells (and doing testing with those), and then iteratively repeating until the bottom is reached.

justinseanmartin commented 6 years ago

@smileyborg - Thanks for the context and completely understand wanting to prevent devs from making a common mistake.

I don't think the proposed workaround is a workable solution. This requires us to either always scroll to the top, or make guesses about which direction to scroll. I think it might break in some cases in cases of infinite scrolling based UIs. Also, making the act of finding an element on the screen rely on UI interaction would add a significant amount of time just to determine if there is a matching element, which would horribly degrade the amount of time it takes to run a test.

Consider the scenario where an element is being loaded from a server and will show up in .25s. Now the test must wait for multiple rounds of scrolling any tableviews on the screen to find out if they contain the element that we're looking for, before the element that we're looking for appears.

Maybe we could short circuit the logic for scrolling tableviews without an explicit opt-in from the test actor (e.g. add a [KIFUIViewTestActor usingTableViewScrolling] method or something), but that still seems a bit gnarly.

I played around with XCUI a little bit over the weekend to see how they handle it, and it appears they do all enumerations via the accessibility hierarchy just fine, which might be sufficient for our case as well.

I still need a reproduction case to investigate this further. Running our full test suite on Xcode 9 + iOS 11 doesn't hit this failure mode.

ghost commented 6 years ago

@justinseanmartin I'm getting this same issue, I think the issue might be related to the following:

// Get the cell directly from the dataSource because UITableView will only vend visible cells
UITableViewCell *cell = [tableView.dataSource tableView:tableView cellForRowAtIndexPath:indexPath];

In theory, if the cell is already on the screen, then that means dequeueReusableCellWithIdentifier:forIndexPath: has already been called, so the code block above would call that a second time, triggering the assertion @smileyborg mentioned.

I would assume a valid solution is to first query the table's cellForRowAtIndexPath method directly, then if it returns nil, query the dataSource's cellForRowAtIndexPath method.

ghost commented 6 years ago

Actually reading into the code more it appears this is already handled slightly above:

// Skip visible rows because they are already handled
NSIndexPath *indexPath = [NSIndexPath indexPathForRow:row inSection:section];
if ([indexPathsForVisibleRows containsObject:indexPath]) {
    continue;
}

Not entirely sure why it's failing, unfortunately I can't reproduce it in a sample project. Most likely it's something to do with the complexity of the view hierarchy.

justinseanmartin commented 6 years ago

I have seen issues in the past where the set of indexPathsForVisibleRows contains some elements that are actually offscreen (but only slightly) and thus KIF fails to scroll them into view and tap on them. Don't think that is related though.

mikelupo commented 6 years ago

@justinseanmartin @smileyborg Any chance you guys could pull this branch from my repo, addTestThatBreaksIOS11. I need a little help trying to reproduce this exception in a test case.

Thanks in advance

justinseanmartin commented 6 years ago

One guess is if it is caused if some elements are loaded into the table view after some delay, so KIF doesn't find it on the first element scan? Just a guess though.

mikelupo commented 6 years ago

Thanks @justinseanmartin. Still interested if anyone can modify what I posted in that branch and make it fail. I tried on both the simulator and on a real device a few different scenarios. To no avail.

inPhilly commented 6 years ago

Any developments here?

justinseanmartin commented 6 years ago

There is a potential fix in #1020 that you could try out locally to see how it works for you though. @inPhilly - if you try that, please report back on if it works for you and any impact on your test suite.

I'm still hoping for a sample test showing the exception in isolation that can be investigated. I haven't had time to dig in and experiment, but I'd dedicate more time if there was an isolated reproduction to look at.

edwinlai commented 6 years ago

Just checking in, is there any update on this?

mikelupo commented 6 years ago

Is the patch in #1020 working for you?

skyem commented 6 years ago

I'll say that the patch did work for me to fix this issue, but the step back in KIF version broke other things for me (tapping on collection view cells in a calendar), so I couldn't use it. Will it be merged soon?

edwinlai commented 6 years ago

Yes, the patch works for us.

mikelupo commented 6 years ago

@justinseanmartin ?

edwinlai commented 6 years ago

Bump @justinseanmartin

soleares commented 6 years ago

This issue made KIF totally unusable for our project. It went from "just working" to being really flaky. I'm concerned that users trying KIF for the first time will have a bad experience with it due to this issue.

The fix has worked great for months. I would love to see this merged and am happy to help if I can.

mikelupo commented 6 years ago

My change does not merge cleanly across a few files. I am working on it and hope to finish it today.

justinseanmartin commented 6 years ago

Whoops, I didn't mean to hit close, sorry.

I apologize for going dark on this for so long. I've been busy and haven't really had time to dedicate to this (not a good excuse for a maintainer of an OSS project).

I reran all of our tests with these changes rebased on latest master and it looks like it runs fine. There is some more scrolling that happens than normal, which causes tests to behave a bit differently and breaks some things. That said, all of the issues encountered appear to be our own and not due to the fix. I'll put some comments in the PR, but 👍 to merge the fix.

justinseanmartin commented 6 years ago

Fixed in latest master. If people report this as working for them, I'll make a release.

heitorfr commented 6 years ago

The latest master fixed the issue for me. Thanks

soleares commented 6 years ago

The lastest master fixes the issue for us as well. We've been using mike's branch for a while as a workaround.

justinseanmartin commented 6 years ago

I just published 3.7.4 with this fix in it. Thanks @mikelupo for your hard work on this fix.