Consider genymotion emulator support for android builds

[takn]

Genymotion is a much better emulator than the vanilla emulators. Consider providing support for it.

http://www.genymotion.com/

[BanzaiMan]

Could you elaborate on what makes it "better"?

A third-party emulator support may sound great, but long-term product viability and our own support capability must be considered before making the decision.

[krschultz]

I agree that Genymotion is a great tool for developing code, and I use it instead of the official emulator for that purpose, but I don't think it makes sense for a CI environment.

• Correctness is more important than speed for the CI server. The emulator is a pain locally because it is slow, but the difference in build time between Travis & local is not because of the emulator. Travis (correctly) starts with a clean environment & fetches the dependencies on each build. My current Travis build is about 10 minutes, of which 3-4 is basic setup. After that it is all building & tests. If that were to balloon to 30-40 minutes, then it is a problem, but until then I'm OK with it. And if it did get that long, the answer would be build slaves & parallelizing (features I have used before in Jenkins), not switching to Genymotion. Just yesterday I found a bug in a commit that built fine locally and worked with genymotion, but failed on Travis in an official emulator. That is the point of CI.
• Genymotion doesn't officially match up with what most people are building. For one thing, Google APIs / Google Play are not officially supported by Genymotion. Of course everyone adds it after downloading, but I don't think Travis can do that legally for everyone. Also, Genymotion is going to lag behind the emulators - there is no L Preview option in Genymotion right now.
• Jake Wharton wrote a great piece about why we need a faster way tool for running the apps during development: Android needs a simulator not an emulator. I think he is right. But the reaction from the tools team at Google IO (to paraphrase) was that as Android goes into more and more devices - TVs, Cars, Wear - it becomes difficult to maintain accurate simulators. Emulators just need to be setup for each architecture and then it runs effectively the same SDK. That makes me think Genymotion will only work for a smaller % of all apps in the future.

Given infinite resources, sure Travis should add it. Considering that they have limited resources, I think there are more pressing problems.

[nhaarman]

Correctness is more important than speed for the CI server. The emulator is a pain locally because it is slow, but the difference in build time between Travis & local is not because of the emulator.

Sure, we don't care about the speed of the building process. However, the emulator is so slow, that it causes tests which rely on events occuring within a certain time to fail. Of course these time intervals can be stretched, but this should be a workaround only - I do not plan to add a 30 second time interval to my tests because the Travis run needs to pass as well.

[stephanenicolas]

I have now been using Travis CI with android emulators for more than a year, around 2 even, mostly for the RoboSpice library, but some others too. And yes there are some issues with speed and stability with android emulators. I faced some situations like :

• quite often travis can't connect reliably to the emulator, shell commands appear to be unresponsive. Like in this build (and many others) : https://travis-ci.org/roboguice/roboguice/builds/35758434. It never happened to me with GenyMotion.
• matrix builds take up to 20 minutes before failing, per build for large projects. Yes time is an issue in TDD development in those cases.
• I have seen a lot of race condition, random failure troubles happen only on the super slow emulator that I could not reproduce on GenyMotion or devices. Those failing tests remain, since more than a year, complete mysteries to me. Some builds fail for reasons I can't understand and can't reproduce anywhere else. To give you an example of a case I could solve : the tests are some time so slow to start on emulator that http://developer.android.com/reference/android/test/AndroidTestCase.html#getContext() failed because the test can't set a context fast enough, only on emulator.
• time outs given to succeed or fail a test are far beyond normal to pass the emulator due to its slowness. I would say up to 5 times in some cases.

And slowness may even prevent the adoption of travis, and more generally automated testing for stuff like espresso/ui automator UI testing. It costs so much time to test UI with an emulator and you get so many troubles of stability because of speed that it's really not worth setting up tests.

I acknowledge your arguments @krschultz, but for all these reasons I have been dreaming of getting GenyMotion to work on travis as tests don't fail for the reasons mentioned above with it. Emulator also has drawbacks and is far from perfect for testing.

I even tried to set up genymotion on travis and was stuck when I realized that both of them use a virtualization solution that seem to make it impossible to run genymotion inside a travis build, at least at the user level.

So, I would say that, maybe without being a replacement for the emulator, but having the possibility to choose between the emulator and genymotion would be great for many android FOSS devs.

To give you an idea, I think I have been following any update on this topic regularly, googling out "travis genymotion" monthly for more than a year. That's how I found this thread.. ;)

[krschultz]

@stephanenicolas I definitely put a lot of value in your opinion on this one because I think you have pushed further than anyone else with the Quality-Tools-for-Android project.

I think the absolute first question is if the licensing of Genymotion is workable for this use case. AFAIK, Genymotion is not open source and the license quoted below precludes this use. I have actually purchased a license to get a few of the higher level features, but I would suspect without reaching out to Genymobile that TravisCI (the company) can't just drop the emulator in.

License granted: Limited GENYMOBILE grants the end user or the licensee a non-transferable, non-sublicenseable, non-assignable, terminable, limited, and non-exclusive license to access and use the software application developed by GENYMOBILE.

This license is granted to the end user only and exclusively in connection with personal use, the end user is an individual, and not a professional, who downloads the application for personal and private needs, excluding commercial and professional environment.

The end user and the licensee do not have any right or license to, and will not, directly or indirectly, in any manner whatsoever:

• assign, transfer, sell, encumber or modify the software/application developed by GENYMOBILE or include this applicationin a product whatsoever,
• use or utilize this software/application for any purpose or in any manner that is illegal or in violation of any applicable law or rights of any third party,

Generally speaking, all access, performance or use of the application except as expressly permitted by these terms and any use in accordance to GENYMOBILE’s purpose shall be deemed to be an infringement of GENYMOBILE’s intellectual property rights on its software/application and open all rights and remedies in connection with such infringement.

I'm starting to think that perhaps TravisCI should support end users supplying the worker. I don't think it works for the free tier, but on the paid account, it would be great if I could fork the worker recipe, add my changes, and run it on my EC2 account. I really like the Travis UI & the hooks into various services, but sometimes you need to control the worker VM as you would in Jenkins. This came up with regards to XCode 6, Android L, and now GenyMotion. I think it's a common problem on mobile. I'm not exactly sure how it would work, but I think it is the general purpose solution to the problem. I also think it would be a feature worth charing for, because obviously there is a bit of setup going on there.

[stephanenicolas]

@krschultz thx for your kind words. Android has done good progress now in integrating all quality tools and I don't find time and motivations to maintain the Quality Tools project up to date at the pace of changes in android plugin.

I know the guys from GenyMobile pretty well. Do you want me to put you guys in contact ? I even think a free solution for travis CI could be possible for them, they really wanna support OSS initiatives even if there soft is not OSS.

[krschultz]

@stephanenicolas I'm not affiliated w/ TravisCI, but if those guys want to talk to Genymotion that might make sense for them. Maybe @BanzaiMan can comment on that one.

[vitorprado]

:+1:

[BanzaiMan]

I'm locking this issue for now until we have spare cycles to look into it deeper.