search

stephanenicolas / Quality-Tools-for-Android

1.27k stars 235 forks source link

readme

Quality Tools for Android

<img src="https://raw.github.com/stephanenicolas/Quality-Tools-for-Android/master/gfx/bugdroid-duke-armor.jpg" width="350px" />

This is an Android sample app + tests that will be used to work on various project to increase the quality of the Android platform.

The idea is that Android programming is still in its infancy compared to the Java world. The Android community needs more robustness in Android apps and it looks like a good idea to build on the Java world experience and use its best tools for Quality Analysis.

We want to provide a full featured industrial development environment that can be used to create more robust projects on Android, by using any of the most interesting and popular technologies.

Here are some slides to present Quality Tools for Android.

Already integrated :

What is missing (TODO/INTEGRATE) :

  1. get aggregated tests and code coverage for all testing technologies inside a nice Sonar dashboard for Android.
  2. add support for monkey runner through maven
  3. add calabash support.
  4. Add support for JUnit 4 on Android : http://stackoverflow.com/questions/9809180/why-is-junit-4-on-android-not-working

Usage

This section describes how to build & test the project using those different testing technologies.

Please note that this project is under active development. Some goals may require a snapshot version of the maven android plugin available on sonatype snapshot repo.

Install Android Latest SDK through Android SDK Manager

This can be done graphically, or via command line (for CI servers).

Install the Android SDK through maven-android-sdk-deployer

As it takes time to get android jars in maven central, including android UI automator jars in maven central, we recommend to use maven-android-sdk-deployer to obtain android artefacts. This step can also be executed on a CI server.

#install Android SDK 17 local files to local maven repo  
git clone git@github.com:mosabua/maven-android-sdk-deployer.git
cd maven-android-sdk-deployer/
mvn install -P 4.2
#Add V4 support library (to use FEST Android)
cd extras/compatibility-v4/
mvn install

Standard Android testing APIs and code coverage using emma

To build the sample project and run the sample app on a plugged rooted device / running emulator : 

# in parent folder
mvn clean install -P emma
mvn sonar:sonar -P emma

you will get tests results in : target/surefire-reports/. you will get tests coverage in : target/emma/.

Here is the result in sonar :

You may need to restart adb as root to be able to pull the emma coverage file. In a terminal, type :

adb root

Robolectric and code coverage using cobertura

# in parent folder
mvn clean cobertura:cobertura -P cobertura
mvn sonar:sonar -P cobertura

Here is the result in sonar :

Unified code coverage for both Robolectric and standard Android Junit tests via Jacoco

Using offline instrumentation of Jacoco, it is possilbe to completly replace emma by jacoco for instrumentation. This allows to get both robolectric and standard tests code coverage inside the same project dashboard with sonar.

# in parent folder
mvn clean install -P jacoco
mvn sonar:sonar -P jacoco

Here is the result in sonar :

UI Automator

# in parent folder
mvn clean install -P uiautomator
mvn sonar:sonar -P uiautomator

Here is the result in sonar :

Espresso

To build the sample project and run the sample app on a plugged device / running emulator : 

# in parent folder
mvn clean install -P espresso

Spoon from Squareup

# in parent folder
mvn clean install -P spoon

#then browse to android-sample-tests/target/spoon-output/index.html

Here is the result in a browser :

Monkey testing

Monkey is part of Android SDK and allows to harness Application UI and test their robustness. We contributed to a new maven android plugin goal to use monkey automatically and get reports in junit format.

The results can be displayed inside sonar and will appear as normal unit tests.

# in parent folder
mvn clean compile -P monkey
mvn sonar:sonar -P monkey

Here is the result in sonar :

Package cycles check via classycle

You will need a JDK 1.7 for this profile to work correctly.

# in parent folder
mvn clean compile -P cycle

Will check package cycles (also called package tangling in Sonar) and check the build if given cycles are detected. Classycle lets you define architectural constraints that can be checked automatically.

Depedency definition files are very simple to edit. Here is an example : 

show allResults

###define packages / groups of packages of interest

## layers
[ui] = com.octo.android.sample.ui.*
[other] = com.octo.android.sample.* excluding [ui]

###check layers integrity
check [other] independentOf [ui]

Robolectric development in eclipse

RoboElectric tests are separated from the sample project, as all testing technologies.

To make this configuration work in eclipse, do the following :

//TODO update this

TODO : POST a SNAPSHOT of the JUnit run config in eclipse

Now, simply execute your project as a JUnit project and all robolectric tests will get executed.

Using Gradle

All gradle-related file are stored in folder gradle.

With Gradle 1.8+ and android gradle plugin 0.6.+ :

build the app under tests

# in parent folder
gradle clean assemble

launch the app under tests

# in parent folder
gradle :android-sample:installDebug

play standard android tests (without emma coverage):

# in parent folder
gradle clean assembleDebug connectedInstrumentTest
#export to sonar
gradle :android-sample:sonarRunner

play espresso tests (without emma coverage):

# in parent folder
gradle clean assembleDebug :android-sample-espresso-tests:connectedInstrumentTest

play robolectric tests :

# in parent folder
gradle clean assembleDebug robolectric
#export to sonar
gradle :android-sample-robolectric-tests:sonarRunner

Findbugs + Checkstyle + PMD + classycle :

# in parent folder
gradle check

or independently : 

# in parent folder
#you need to run assemble before most of those
#gradle assembleDebug
gradle checkstyle
gradle findbugs
gradle pmd
gradle classycle

Running lint :

# in parent folder
gradle :android-sample:lint

Aggregate reports :

# in parent folder
gradle buildDashboard

TODO run test coverage using emma (standard tests):

TODO run test coverage using cobertura (robolectric tests):

TODO run test coverage using jacoco (both tests):

TODO play uiautomator tests

TODO play monkey tests

TODO play monkey runner tests

TODO shoot more stuff to sonar

Thanks to

Quality Tools for Android in the news !!

License

Copyright (C) 2013 Stéphane Nicolas & Jérôme Van Der Linden

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

     http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,