google / EarlGrey

:tea: iOS UI Automation Test Framework
http://google.github.io/EarlGrey/
Apache License 2.0
5.62k stars 741 forks source link

EarlGrey 2.0 | Cocoapods | Can't run target app without UITests #971

Closed konrad-zdunczyk-luxmed closed 3 years ago

konrad-zdunczyk-luxmed commented 5 years ago

Target app crashes on start if run alone (without UITests ex. by hitting cmd+R) and get

Fatal failure: EarlGrey's app component has been launched without edoPort assigned. You are probably running the application under test by itself, which does not work since the embedded EarlGrey component needs its test counterpart present. in /Users/brettfazio/dev/reducingimports/EarlGrey/AppFramework/DistantObject/GREYHostApplicationDistantObject.m:93

It can be reproduce in Example from earlgrey2 branch.

Xcode 10.2.1 Swift 5.0 Cocoapods 1.6.1

RichardGuion commented 5 years ago

I ran into the same exact error this week.

tirodkar commented 5 years ago

Can you do a clean build and then run it? Basically, you're running it on a version of your app with EG's AppFramework still linked in.

RichardGuion commented 5 years ago

I reproduced on the EarlGreyExample for 2.0. When it run the app standalone I get the error described above. I cleaned the build folder, even deleted DerivedData, then hit Run to install on the simulator. Same crash. Clean then Build for Running, same crash again.

jainhitesh3 commented 5 years ago

Any update on this? I'm also seeing this similar issue, I tried cleaning derived data (also deleted deriveddata directory). But still no luck.

tirodkar commented 5 years ago

EarlGrey's app component requires the test component when launched. That is what is causing is crash. The application has DYLD_INSERT_LIBRARIES loaded into its app environment which tries to launch EarlGrey when you run it by itself.

RichardGuion commented 5 years ago

I see that in the code here: https://github.com/google/EarlGrey/blob/earlgrey2/TestLib/XCTestCase/XCUIApplication+GREYEnvironment.m#L23

I had to read up on DYLD_INSERT_LIBRARIES. Very cool how this works.

But I cannot explain why the app still crashes running standalone on a clean build without the tests. I created a separate scheme just for the tests and the app together. I had a second scheme with just the app itself, no tests, no EarlGrey. Even after cleaning DerivedData and building from scratch, I still received the error.

jainhitesh3 commented 5 years ago

Hi @tirodkar : When can we expect a fix/workaround for this issue?

tirodkar commented 5 years ago

Is the bundle id the same for both apps? Can you please ensure they are different?

RichardGuion commented 5 years ago

@tirodkar I looked at the Pods project and saw that they are different...is this where you wanted us to look?

EarlGreyApp bundle ID EarlGreyTest bundle ID
tirodkar commented 5 years ago

No. I was talking about your application being tested.

On Tue, Oct 1, 2019 at 11:01 AM Richard Guion notifications@github.com wrote:

@tirodkar https://github.com/tirodkar I looked at the Pods project and saw that they are different...is this where you wanted us to look?

[image: EarlGreyApp bundle ID] https://user-images.githubusercontent.com/4001013/65987421-9d4fce80-e43a-11e9-8cea-90b91551b6ec.png

[image: EarlGreyTest bundle ID] https://user-images.githubusercontent.com/4001013/65987436-a5a80980-e43a-11e9-84dd-86a28dda7636.png

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/google/EarlGrey/issues/971?email_source=notifications&email_token=ADRQEFKRW5HUNTLF3ZY2XE3QMOF6PA5CNFSM4IBLYR32YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEACFW7A#issuecomment-537156476, or mute the thread https://github.com/notifications/unsubscribe-auth/ADRQEFI35IG4OXYSAXM3UVTQMOF6PANCNFSM4IBLYR3Q .

RichardGuion commented 5 years ago

I'm just using the EarlGrey demo, which reproduces the problem. The bundle id for the app vs the tests are different.

EarlGreyExampleSwift bundle EarlGreyExampleSwiftUITests bundle
RichardGuion commented 5 years ago

@brettfazio - wondering if you could give us any insight here?

The problem: Cocoapods installs EG2 just fine and allows us to run the tests. However the app itself can no longer be run stand-alone, because the app links in AppFramework.framework directly all the time. This leads to the error message when launching the app (outside the UI tests) "EarlGrey's app component has been launched without edoPort assigned." We can all reproduce this using the EG2 demo EarlGreyExample in the repo.

I see this in the podspec for EarlGreyApp: https://github.com/google/EarlGrey/blob/earlgrey2/EarlGreyApp.podspec#L19

This translates to the linker setting we see in the project for the EarlGreyExampleSwift app in the demo after we do 'pod install'

Cocoapod app target linker setting

With this setting you can see, any time the app is launched, this component is part of the app, and the missing edoPort throws that exception I mentioned above, when executed without the companion test app.

I was wondering why we don't do the Copy Files method in the setup description for the cloning repo method in the setup docs: https://github.com/google/EarlGrey/blob/earlgrey2/docs/setup.md

Copy AppFramework

This method leaves the app target alone and copies over AppFramework.framework after the test target is built.

Regardless, with the pod version, I did an experiment. I removed the linker setting for AppFramework.framework in the app target.

xcconfig file remove appframeworks

I removed it from Pods-EarlGreyExampleSwift.debug.xcconfig. Cleared my DerivedData folder, rebuilt the tests, and the tests ran fine. But I was also, for the first time, able to launch the EarlGreyExampleSwift app standalone!

linker flag AppFramework gone

Not sure why it works...but it does. I did double check to make sure I am still running in Debug mode.

brettfazio commented 5 years ago

@RichardGuion If you take a look at either of the Podspecs neither of them add the "-Objc" flag or the "-framework "AppFramework"" flag. So I believe that to be added automatically.

RichardGuion commented 5 years ago

@brettfazio yes - I think it is added by

s.vendored_frameworks = "AppFramework.framework"

brettfazio commented 5 years ago

@RichardGuion the line you pointed out here is necessary because without it the Framework would not be linked in. So I don't believe that to be the direct culprit there.

If the goal here is to only have AppFramework link on one of the two, in the case you pointed out not having it link on "Debug", I believe that https://guides.cocoapods.org/syntax/podfile.html#project could be of some use.

There is stuff in there that speaks on ":Debug" and ":Release" so potentially a modification to EarlGreyApp.podspec could be made using those project custom build configurations.

jainhitesh3 commented 4 years ago

Hi @brettfazio / @tirodkar: Is there any update on this? Is there a plan to pick this issue anytime soon? If not could you please tell if there is any workaround that can be done to solve this?

tirodkar commented 4 years ago

@jainhitesh3 this is by design with EG. We have to embed EarlGrey into your application for testing it.

When you add the EarlGreyApp component to your application and then generate the CocoaPods workspace, your entire application target now contains the App component inside it. Regardless of a clean build, deleted derived data etc - every time you launch the application, it is launching with any EG related flags / binaries that have been added in the Podfile.

As @RichardGuion showed - you could modify the Build Settings in the workspace and get your application to launch by itself. You can then do a new pod update to run the tests. Alternatively, you can duplicate your application target and use that as the Target Host with your EG tests.

jainhitesh3 commented 4 years ago

@tirodkar: I tried the steps mentioned by @RichardGuion. I'm able to run my target app independently (without launching any tests).

Also as suggested by @brettfazio in the above comment, is it possible to update EarlGreyApp.podspec file, and specify if we need this framework for ":Debug" or ":Release"?

jangbn commented 3 years ago

Is there any update on this? Still crash when running the app standalone.

EarlGrey 2.2.0 Xcode 12

tirodkar commented 3 years ago

With 2.2.0 - we have an update on this. You can specify EarlGreyStandaloneMode: 1 in your Scheme's Environment Variables and you can run your application with EarlGrey's app component linked in.

christianmierez commented 3 years ago

@tirodkar great to know this was fixed! It'd be also helpful to update the setup doc with this information.