KIF loops through NSFetchedResultsController results after view dismissed

[babbage]

Summary of the problem When one of my KIF tests finishes any of my KIF tests leave a view that had a UIContainerView with a UICollectionView inside it, I am seeing looping through all of the entities in the NSFetchedResultsController fetch request that was made on that previous view that has now been dismissed. I am seeing this in all my tests that interact with such views, but in most cases there were few enough results that it was not noticeable previously. In one case, however this means that all 600 results are fetched or more importantly the methods being called make it appear to me like all 600 UICollectionViewCells are being rendered offscreen (or at least created/dequeued and values set), rather than the just 10 that were initially fetched in the original functional view. Normally, this fetch request and the results would have already been deallocated.

The detail I have a series of functional tests set up with KIF. One of them examines a number of functional behaviours in a view that has a search box at the top, a UIContainerView below, and a UICollectionView loaded in the container view. Initially, that UICollectionView shows all results (about 600), though the data source is an NSFetchedResultsController so only about 10 are loaded initially for the three results that can appear on screen in this design.

This particular functional test is not actually doing any searching, just looking at what appears alphabetically at the top of the list. The test is getting references to various aspects of the view, in order to compute whether subview frames are falling entirely within their parent views etc, so the purpose of it is to do some metrics on the visible view, when it is displayed in a functional context. (Actual searching is tested in other KIF tests that are passing fine.)

This particular test passes fine if it is run solely by itself—in that circumstance, the looping is not seen. However, when any test is run after it, the looping is seen. It frequently causes the following test to timeout, as it blocks the UI for more than 10 seconds. There are things I can do to optimise the code, that might prevent this UI blocking (and I will do that in due course) but the key issue here is that that would just be plastering over what seems to be a problem with this test... and isn't an actual performance issue in the use of the app that is significant at this point in development.

If I remove all of the content of this test, other than it's setup, the problem does not occur. I am struggling to tie it down to particular content within the test, as every time I think I've found something that fixes this, it proves to be a temporary change, and the problem re-appears. I've also seen a number of other times when the problem has "failed" to occur, but I can't seem to tie this down to something predictable like not occurring the first time after a build, or a clean build. Would much prefer to have my test suite predictably working rather than just ignore this or kludge around it, but am pulling my hair out here.

Oh, and finally, on some occasions (for reasons I've not been able to determine) it seems happy to loop away for longer than the 10 seconds that would normally time out for the next test trying to wait for a button to tap, and in those cases I've repeatedly seen from logging that it loops exactly three times through the results of the fetch request, before stopping.

Can anyone advise what might be a way to mitigate this, or provide a pointer to how I can tie this down further?

[babbage]

Have determined now that in fact this looping is occurring in every case where KIF leaves a view that contained a UICollectionView inside a UIContainerView. (Don't know if the container view is significant, as all my UICollectionViews are in container views in this app.) I've tried to mark changes above to correct the information, without erasing the history of the original post. What I've realised is that in other cases where the total fetch results only returned 20 results, looping through this result set wasn't obvious until additional logging demonstrated this was happening. In the one test where there are 600 results, things were obviously locking up and timing out.

It looks like even though the simulator is now displaying a new view on screen, KIF continues to be directing its own attention to the previous view, e.g., looking for a view with a particular label among the items of a previous view that is no longer on screen first, rather than among the elements that are currently being presented to the user. If it does not find the view on the previous screen, it then starts looking on the current screen. But this means that "waitForViewWithAccessibilityLabel" is doing anything but waiting, and rather would be better named "searchAggressivelyForViewWithAccessibilityLabel". I'm not seriously suggesting that the method be renamed, but it seems to me that this is what is occurring, insofar as it is as a result dequeuing hundreds of UICollectionViewCells that otherwise had not been created functionally.

Perhaps I can avoid this by actually not including these waitForViews, which I assumed were going to ensure that the next functional test wouldn't start until the transition from the previous view had completed—as instead it is actually delaying the transition from a previous view (sometimes visibly in the simulator, but sometimes only in KIF's focus while the simulator moves on). There may well be established KIF patterns that I'm just not aware of and not following here, so pleased to be advised.

[babbage]

OK. The issue is, if you have a fetchedResultsController that could potentially fetch a lot of items if you scrolled, then any method that involves the absence of a label is a bit of a black hole. In such circumstances, waitForViewWithAccessibilityLabel: (when that label is expected to be in the next scene, not in a current view) and waitForAbsenceOfViewWithAccessibilityLabel: can cause the problem described above.

I had adopted a pattern of including a check at the end of a test that ensured the test had returned to the home screen of the app before the test finished, so that the next test would be starting at a predictable location. Unbeknownst to me, using waitForViewWithAccessibilityLabel: to look for a label on that home screen would cause the kind of cycle described above where KIF would search for the label through all possible results of the fetch, something that continued to search through those fetch results even when in the UI of the simulator the app had exited out to the home screen. Similarly, waitForAbsenceOfViewWithAccessibilityLabel: would cause the same cycle.

Inserting a half second pause via [tester waitForTimeInterval:0.5]; before a final [tester waitForAbsenceOfViewWithAccessibilityLabel:@"label"] eliminates the problem, while still ensuring the test does not actually end until the simulator has exited correctly back to the home screen of the app.

It seems to me that this is a bit of a design flaw or at least design trade-off in KIF. One possible way to mitigate most of the effects of this would be if KIF only cycled through x number of labels in a view (50? 100? something known to not be unreasonable) after which it paused for 0.5 seconds and then restarted it's search. Admittedly, this is still a bit of a kludge but I think it would likely eliminate the functional effects of this issue in almost all situations. It would affect any timed performance testing a little. I'm not sure if KIF is even being used for that? Wouldn't seem ideal. I would be happy to look at implementing this, but only if such a change was considered acceptable. (I can imagine it might not be.)

Ideally, it would do this only in situations with an NSFetchedResultsController or where there were known or predicted to be an extremely large number of accessible UI elements. Not confident there'd be a suitable way to do this.