google / EarlGrey

:tea: iOS UI Automation Test Framework
http://google.github.io/EarlGrey/
Apache License 2.0
5.61k stars 741 forks source link

Help! We really really reeeeeeaaaaaaaaally need EarlGrey 1.17 #1337

Open OuterCloud opened 4 years ago

OuterCloud commented 4 years ago

God. help! Really Really Need Help! Hope EarlGrey 1.17 is comming soon!!!!!!!!!!!!! EarlGrey 2.0 is not what we want. (Our test cases are based on XCTestCase)


Below is the problem I met and can not resolve:

EarlGrey 1.16 running on iPhones(not simulator) >iOS 13.2 system(such as iOS 13.2.2, iOS 13.3 and so on) will find this error when running the test case based on XCTestCase (see the Exception Details below, please): Exception Details: Timed out waiting for app to idle.

Please check the fatal error below:


Exception: TimeoutException

Exception Name: TimeoutException Exception Reason: Couldn't assert that ([GREYCondition conditionWithName:@"等待条件满足" block:^BOOL{ return [[QNReadSomeManager sharedInstance] someHistoryList].count == 0; }]) is true. Exception Details: Timed out waiting for app to idle.

tirodkar commented 4 years ago

@OuterCloud we are not doing a new EarlGrey 1.17 release as we've deprecated it for over a year now. Can you tell us more on why EarlGrey 2.0 is not a good fit for you? I'll be happy to take in any contributions from your side in order to make it better for your use case.

Exception: TimeoutException

Exception Name: TimeoutException
Exception Reason: Couldn't assert that ([GREYCondition conditionWithName:@"等待条件满足" block:^BOOL{ return [[QNReadSomeManager sharedInstance] someHistoryList].count == 0; }]) is true.
Exception Details: Timed out waiting for app to idle.

Is EarlGrey failing to idle here? It looks like the failure is in a condition (which is more or less a loop with a runloop drain) and [QNReadSomeManager sharedInstance] someHistoryList].count which seems internal to your code.

OuterCloud commented 4 years ago

@tirodkar We have a large amount of test cases, about 2000 cases all together. The cases are based on XCTestCase. If we use EarlGrey 2.0, we can not use XCTestCase any more, and we can not use tools like OCMock, and can not import class in app conveniently

OuterCloud commented 4 years ago

For example, we have a lot of cases using code from app directly by import the class and methods and properties to do the test. We have a lot of methods called directly from app and we get the properties of the objects from app directly to do some assert. How can we use EarlGrey 2.0 without changing our XCTestCase framework? This is a big problem. image

OuterCloud commented 4 years ago

We navigate to a page in app directly by using url or schemes. And then, we get the top view controller. And then, we call the methods in the controller or use the properties in the controller to do some asserts. In this progress, we can even use OCMock to partial mock the object in controller to do some method's expect, call and verify. EarlGrey 1.0 is based on XCTestCase, this is a big reason we choose this framework. Because we need XCTestCase to do a lot of test based on the code of app.

OuterCloud commented 4 years ago

@OuterCloud we are not doing a new EarlGrey 1.17 release as we've deprecated it for over a year now. Can you tell us more on why EarlGrey 2.0 is not a good fit for you? I'll be happy to take in any contributions from your side in order to make it better for your use case.

Exception: TimeoutException

Exception Name: TimeoutException
Exception Reason: Couldn't assert that ([GREYCondition conditionWithName:@"等待条件满足" block:^BOOL{ return [[QNReadSomeManager sharedInstance] someHistoryList].count == 0; }]) is true.
Exception Details: Timed out waiting for app to idle.

Is EarlGrey failing to idle here? It looks like the failure is in a condition (which is more or less a loop with a runloop drain) and [QNReadSomeManager sharedInstance] someHistoryList].count which seems internal to your code.

Yes, [QNReadSomeManager sharedInstance] someHistoryList].count is code from our app. EarlGrey always timeout waiting for app to idle.

OuterCloud commented 3 years ago

We need EarlGrey 1 support more. We need EarlGrey going on.

radhap commented 3 years ago

This is exact reason, we chose Earlgrey1 in the first place for whiteBox testing.. with different mocks but now we are not abke to continue because Earlgrey1 does not support Xcode12/iOS 14 due to https://github.com/google/EarlGrey/issues/1447

tirodkar commented 3 years ago

@OuterCloud Can you try [GREY_REMOTE_CLASS_IN_APP(QNReadSomeManager) sharedInstance] someHistoryList].count? You can still do white-box testing with eDO and EarlGrey. There's a new layer now that allows you to do it.