Simplifying the comparison of existing tools for mining android sandboxes.
Due to the popularization of Android and the full range of applications targeting this platform, many security issues have emerged, attracting researchers and practitioners' attention. As such, many techniques for addressing security Android issues have emerged, including approaches for mining sandboxes using dynamic analysis tools (i.e., automated testing tools).
Undoubtedly, the resulting sandboxes' efficiency depends on the test case generation tools used in the mining procedures; and previous research studies have compared Android test case generation tools for this specific goal.
However, it's difficult to increment the research in this field because reproducing these previous empirical studies is a challenging and time-consuming task. This difficulty occurs because it's necessary to integrate test generation tools that often require different and conflicting versions of the Android platform, programming languages (e.g., Python 2 and Python 3), and software libraries.
To mitigate this issue, we present our droidxp/benchmark, a software infrastructure that allows researchers (and tools developers) to integrate and compare test case generation tools for mining sandboxes.
We evaluated droidxp/benchmark through a reproduction study of previous research work, though considering additional test case generation tools. Our experiment suggests that droidxp/benchmark simplifies the comparison of existing tools for mining sandboxes.
This project depends on:
It's assumed that some commands are in your PATH environment variable: adb
, java
, jarsigner
,
emulator
and aapt
.
Installation scripts can be found in droidxp/benchmark-vm.
Each folder contains a config.sh
script that will install this project's dependencies on the OS
identified by the respective folder's name.
To install the project dependencies:
pip install -r requirements.txt
To run the benchmark, execute the following command:
python main.py
The benchmark controll arguments follows:
usage: python main.py [-h] [--list-tools] [-tools TOOLS [TOOLS ...]] [-t T [T ...]]
[-r R] [--list-outputs] [--output OUTPUT] [--debug] [--version]
optional arguments:
-h, --help show this help message and exit
--list-tools list available tools
-tools TOOLS [TOOLS ...]
List of test tools used in the experiment
-t T [T ...] (-t -time) Threshold of the execution time in the experiment (in seconds)
-r R (-r, -repetitions) Number of repetitions used in the experiment
--list-outputs List available output formats
--output OUTPUT OUTPUT FORMAT that will be used to show results (default: basic)
--debug Run in DEBUG mode (default: false)
--version Print the benchmark version
Here is a sample on how to use the benchmark arguments:
python main.py -tools monkey droidbot -t 60 -r 3
All apps used in this research are listed in LargeI.csv and SmallE.csv files. They can be downloaded using the Python script getApps.py.
These apps come from the Androzoo repository, so it is necessary to acquire an access key and configure it in the Python script:
key="Insert here your key"
These apps MUST be stored in the benchmark/data/input/
folder.
We recommend using the script to download the apps because it formats the name of the files to organize in malign and benign apps.
mkdir tools/bananadroid
touch tools/bananadroid/\_\_init\_\_.py
touch tools/bananadroid/tool.py
execute_tool_specific_logic
function
from ..tool_spec import AbstractTool
class ToolSpec(AbstractTool): ''' AbstractTool.init(self, name, description, process_pattern) Args: name(str): The name of the tool description(str): The tool's description (such as test case genration, and so on) process_pattern(str): A string with the pattern of the processes to be killed after execution ''' def init(self): super(ToolSpec, self).init('bananadroid', 'Bananadroid tool description', 'org.bananadroid')
''' This is our hook method, an extention point that every tool developer must provide an implementation. ''' def execute_tool_specific_logic(self, file_name, timeout):
pass
3. All set, you can test if the benchmark is able to recognize your tool with command
```shell
>>> python main.py --list-tools
INFO:root: [Listing available tools]
[bananadroid] Bananadroid tool description
[...]
To run the benchmark with the droidbot tool, the following dependencies must be satisfied.
Python
(both 2 and 3 are supported)Java
Android SDK
platform_tools
directory in Android SDK to PATH
OpenCV-Python
if you want to run DroidBot in cv mode.Clone the droidbot fork at https://github.com/droidxp/droidbot-stable.git
and install it with pip:
git clone https://github.com/honeynet/droidbot.git
cd droidbot/
pip install -e .
If successfully installed, you should be able to execute droidbot -h
.
More info at the README.md from the fork https://github.com/droidxp/droidbot-stable
.
To run the benchmark with the droidmate tool, the following dependencies must be satisfied.
Java Development Kit (JDK) 8
Android SDK
gnuplot
5.0+
pdf
terminal: brew install gnuplot --with-cairo
JAVA_HOME
ANDROID_HOME
gnuplot
in PATH
More info in the wiki https://github.com/uds-se/droidmate/wiki/Building#environment
.
Clone https://github.com/yzygitzh/Humanoid.git
. To run the benchmark with the Humanoid tool, the following dependencies must be satisfied:
Clone https://github.com/droidxp/sapienz.git
. To run the benchmark with the Sapienz tool, the following dependencies must be satisfied
sudo apt-get install libfreetype6-dev libxml2-dev libxslt1-dev python-dev python-tk
brew install coreutils for gtimeout
sudo pip install -r requirements.txt
at Sapienz root folder.Android SDK 19+
It is also necessary to add the project folder in SAPIENZ_HOME
environment variable. Something like:
export SAPIENZ_HOME=~/workspace-droidxp/sapienz/
Note: The environment variable SAPIENZ_HOME needs to end with a '/' character.