search

google / EarlGrey

:tea: iOS UI Automation Test Framework
http://google.github.io/EarlGrey/
Apache License 2.0
5.62k stars 742 forks source link

EarlGrey 1 iOS13 compatibility? #1015

Open szymon-kazmierczak opened 5 years ago

szymon-kazmierczak commented 5 years ago

There are multiple issues caused by iOS 13 (and already addressed in master branch). When can we expect an iOS 13 compatible version of EarlGrey 1?

tirodkar commented 5 years ago

We are not adding iOS 13 support from our side and are focussing our efforts on EarlGrey 2.0. We do welcome external contributions for white-bo Pods support with it.

szymon-kazmierczak commented 5 years ago

I can understand that, but I checked out latest master and built the project, and it looks good on iOS 13. This can't be said about current released version though (app freezes rendering the tests unusable).

The only issue I've noticed so far is that the keyboard is unable to find the number input button, i could look into fixing that if there's noone maintaining 1.x

Would it not make sense to at least release what you have?

RichardGuion commented 5 years ago

@szymon-kazmierczak I feel like you do, wish the EarlGrey 1.x version could be updated forever. I like how it works in-process along with the app, that has worked well for me. From the EG founders perspective I can see they want to move forward with the 2.0 version and supporting the 1.x version would only hold them back. Time is finite.

However, even if they fixed it for iOS 13, what about iOS 14/15/16? Eventually there has to be a cutoff point and you wouldn't want to continue to write tests for EG 1.x if it falls apart in 6 months. The only way it could work is if there was a significant number of people who could maintain a branch/fork of the EG 1.x code base.

szymon-kazmierczak commented 5 years ago

I totally understand that EG2 is the main priority now, but the lack of communication about EG1 deprecation, and iOS13 fixes merged to EG1 made me (and others) think that iOS 13 support is on its way.

No ill feelings toward anyone! Just hoping to communication could be clearer.

tirodkar commented 5 years ago

We understand. EG 1 has not been in active development for a while. I know there is a breakage with iOS 13. I can work on doing one more Pods release for it. However certain API's like shake device etc. will be broken.

marcosgriselli commented 5 years ago

We understand. EG 1 has not been in active development for a while. I know there is a breakage with iOS 13. I can work on doing one more Pods release for it. However certain API's like shake device etc. will be broken.

@tirodkar will there be a last release for iOS 13? Even if it doesn't support all the APIs. Just so we know if we can delay the migration to EG 2.

tirodkar commented 5 years ago

I was planning to do one last release to prevent breaks. Can you test if the current EG 1 project at HEAD works for you?

marcosgriselli commented 5 years ago

EG 1 project at HEAD works as expected on Xcode 11.0 (11A420a)

marcosgriselli commented 5 years ago

Any updates on this @tirodkar?

Hsilgos commented 5 years ago

There are multiple issues caused by iOS 13 (and already addressed in master branch). When can we expect an iOS 13 compatible version of EarlGrey 1?

Hi, recently I tried to migrate to xcode 11.1 and consequently to iOS 13 and found that EG 1.15.1 hangs in GREYAssert... It's it one of problem which is addressed in master? I see the only 4 commits, where 2 of them just correct typos...

tirodkar commented 5 years ago

@Hsilgos what is the issue that you're getting? EarlGrey 1.0 can fail with iOS 13. Please look at moving to EarlGrey 2.0.

I plan to create one last EarlGrey 1.0 pods release once I have bandwidth but that will only contain a breakage fix. You might see other issues as well.

Hsilgos commented 5 years ago

@Hsilgos what is the issue that you're getting? EarlGrey 1.0 can fail with iOS 13. Please look at moving to EarlGrey 2.0.

I plan to create one last EarlGrey 1.0 pods release once I have bandwidth but that will only contain a breakage fix. You might see other issues as well.

I just create view from storyboard and check that it's not nil with GRAYAssertNotNil and this assert hangs as far as I understand because app is never idle. This works with XCode 10.3 and iOS 12.x but not with XCode 11.1 and iOS 13.

If course we will have to switch to EarlGrey 2.0, but we need to check if we can adopt it to our build system. We use CMake and it works well with EG 1 because it can be built as framework, but EG 2 doesn't support this yet and I afraid that it mean that we need to write CMakeLists for EA 2 as well.

Update The code roughly looks like this:

let storyboardName: String = "EmptyViewTest"

let storyBoard = UIStoryboard(name: storyboardName, bundle: Bundle(for: CurrentClass.self))

let viewController = storyBoard.instantiateInitialViewController()
let window = UIWindow()
window.rootViewController = viewController
window.makeKeyAndVisible()

let view = viewController.view

GREYAssertNotNil(view, reason: "Current view is nil")

and the stack for EarlGrey 1.15.1 is:

0 0x00007fff5182322a in mach_msg_trap ()

1 0x00007fff5182376c in mach_msg ()

2 0x00007fff23b0caf5 in __CFRunLoopServiceMachPort ()

3 0x00007fff23b076ed in __CFRunLoopRun ()

4 0x00007fff23b06e66 in CFRunLoopRunSpecific ()

5 0x000000010eefb9da in -[GREYRunLoopSpinner grey_drainRunLoopInActiveModeAndCheckCondition:forTime:] at /project/external/earlgrey/upstream/EarlGrey/Synchronization/GREYRunLoopSpinner.m:175

6 0x000000010eefb3a5 in -[GREYRunLoopSpinner spinWithStopConditionBlock:] at /project/external/earlgrey/upstream/EarlGrey/Synchronization/GREYRunLoopSpinner.m:58

7 0x000000010ef4864a in -[GREYUIThreadExecutor executeSyncWithTimeout:block:error:] at /project/external/earlgrey/upstream/EarlGrey/Synchronization/GREYUIThreadExecutor.m:227

8 0x000000010ef47b05 in -[GREYUIThreadExecutor drainUntilIdle] at /project/external/earlgrey/upstream/EarlGrey/Synchronization/GREYUIThreadExecutor.m:174

9 0x000000010f1ade6e in GREYWaitUntilIdle() at /project/myapp/ios/tests/src/EarlGrey.swift:98

10 0x000000010f1aca75 in GREYAssert(::details:) at /project/myapp/ios/tests/src/EarlGrey.swift:79

11 0x000000010f1acd91 in GREYAssertNotNil(_:reason:) at /project/myapp/ios/tests/src/EarlGrey.swift:33

...

szymon-kazmierczak commented 5 years ago

What worked in my case was the ios13fixesbranch, not master

Hsilgos commented 5 years ago

@szymon-kazmierczak , thanks i'll try it

tirodkar commented 5 years ago

ios13fixesbranch contains some temp fixes / workarounds for iOS 13 support. I can cut the last Pods release from there. @szymon-kazmierczak are there any other issues that you're seeing with that branch?

szymon-kazmierczak commented 5 years ago

We're seeing some flakiness and issues, but they are probably iOS 13 related - I'll know more early next week once I catch up with other teams.

yongen commented 5 years ago

All our earlgrey tests hang up during app launch after we switch to xcode 11. I try to using ios13fixes branch in our project by modifying the Podfile

pod 'EarlGrey', :git => 'https://github.com/google/EarlGrey.git', :branch => 'ios13fixes'

However, it reports "[!] Unable to read the license file EarlGrey/LICENSE for the spec EarlGrey (1.15.1)"

littlebobert commented 5 years ago

In our Podfile I changed pod 'EarlGrey' to:

pod 'EarlGrey', :git => 'https://github.com/google/EarlGrey.git', :branch => 'ios13fixes'

and ran bundle exec pod update EarlGrey. But that removed "${TARGET_BUILD_DIR}/${FRAMEWORKS_FOLDER_PATH}/EarlGrey.framework" from the Embed Pods Frameworks build phase for our test target and so files in that target can’t import EarlGrey anymore. Here is the relevant section of our Podfile:

target 'UISnapshotTests' do
    use_frameworks! # Required for Swift Test Targets only
    inherit! :search_paths # Required for not double-linking libraries in the app and test targets.
    pod 'EarlGrey', :git => 'https://github.com/google/EarlGrey.git', :branch => 'ios13fixes'
    pod 'iOSSnapshotTestCase'
    test_pods
end

Any ideas?

tirodkar commented 5 years ago

I've moved the fixes to the master branch so you don't need the branching anymore. I'm cutting the release currently.

tirodkar commented 5 years ago

Folks, I have a tentative release out here - https://github.com/google/EarlGrey/releases/tag/1.16.0

Can someone test this before we push it to CocoaPods?

achernoprudov commented 5 years ago

Tested 1.16.0 on ~200 production tests. iPhone 8 (iOS 13), iPhone 11 Pro Max(iOS 13), iPhone SE(iOS 12.2). Everything works, tests pass.

steipete commented 4 years ago

We're using v1 with iOS 13 as well on a few hundred production test, it works really really well.

kikettas commented 4 years ago

Same thing here. We're currently using Earl Grey v1 with iOS 13.3 and Xcode 11.3.1 and it works as before 👍