Deep-GUI is a tool for generating intelligent inputs to test UI-based applications, such as Android or web applications. This document provides instruction on how to train a Deep-GUI model, how to use it in order to automatically interact with an application, and how to quantify the quality of the generated inputs in terms of line coverage.
System requirements:
Before getting started:
~/android-sdk
. Locate sdkmanager
in ~/android-sdk/cmdline-tools/bin
sdkmanager
system-images;android-10;google_apis;x86
using sdkmanager
PATH
/home/$USER
Note: it is important to install the requirements in the exact paths mentioned above.
deep-gui
│
└───apks: a sample set of apk files with their respective emma file.
│
└───configs: configuration templates for different tasks
│
└───models: pre-trained deep-gui models
│
└───scripts: bash scripts used by different parts of the tool
│
└───src: python source files
You can use this software to perform these tasks:
In order to train deep-gui, you need to first collect data.
cd scripts
./create_tester_ref
./clone_avd.sh tester_ref collector_ref
Set the configurations:
In configs/collect-configs.yaml
, set these mandatory values:
collectors: The first number is the number of parallel data collection agents
data_file_dir: The directory where the data is stored in (change this dircetory to collect both training and validation data)
logs_dir: The directory where the tensorboard logs are written to
collectors_apks_path: The directory containing training/validation apks
collector_configs.version_start: If you need to append to existing data, set this number accordnigly
configs/collect-configs.yaml
to src/configs.yaml
cd src; python main.py
cd <logs_dir> # same as logs_dir used in configs.yaml file
tensorboard --logdir=. --reload_interval 1 --samples_per_plugin "images=0"
Connect to localhost:6006
to see tensorboard logs.
To train using the collected data:
Set the configurations:
In configs/train-configs.yaml
, set these mandatory values:
data_file_dir: The directory containing the training data
learner_configs.save_dir: The directory where the trained models are stored. You need to create this directory manually.
learner_configs.validation_dir: The directory containing the validation data
collector_configs.version_start: Set this to a large number
configs/train-configs.yaml
to src/configs.yaml
cd src; python main.py
Note: A pre-trained model is availabe in models
To run the experiments:
Create emulator template
cd scripts
./create_tester_ref
./clone_avd.sh tester_ref collector_ref
Set the configurations:
In configs/monkey-test-configs.yaml
, set these mandatory values:
testers: The first number is the number of parallel agents.
weights_file.e10: The path to the model that is to be used (.hdf5)
Copy configs/monkey-test-configs.yaml
to src/configs.yaml
Run the code:
cd scripts
./run_all_monkies.sh <experiment-name> monkey 0 <num-agents> <num-rounds> <apk-dir> <experiment-dir>
cd scripts
./run_all_monkies.sh <experiment-name> deep 1 <num-agents> 1 <apk-dir> <experiment-dir>
where:
<experiment-name>: An arbitrary name for the experiment
<num-agents>: Number of parallel agents (must match the configs.yaml file)
<apk-dir>: The directory containing test apks. Each apk named app.apk needs to have a emma file in the same directory named app.apk.em
<experiment-dir>: The directory containing the experiment files
After the experiment is completed, run this:
cd ../src
python update_tb.py <experiment-dir> <experiment-name> <experiment-dir>/tb_otest_logs <apk-dir>
Look at the logs:
cd <experiment-dir>/tb_otest_logs
tensorboard --logdir=. --reload_interval 1 --samples_per_plugin "images=0"
Connect to localhost:6006
to see tensorboard logs.
To run the experiments:
Set the configurations:
In configs/web-configs.yaml
, set these mandatory values:
testers: Set to [<num-parallel-agents>, [0, 0, 0, 0, 0, 1], monkey]
reward_predictor: [RandomRewardPredictor, random]
testers: Set to [<num-parallel-agents>, [.7, .3], deep, c99s, e10]
reward_predictor: [UNetRewardPredictor, unet]
logs_dir: The directory where the tensorboard logs are written to
weights_file.e10: The path to the model that is to be used (.hdf5)
browser_configs.apps: The list of websites to be explored
configs/web-configs.yaml
to src/configs.yaml
cd src; python main.py
cd <logs_dir> # same as logs_dir used in configs.yaml file
tensorboard --logdir=. --reload_interval 1 --samples_per_plugin "images=0"
Connect to localhost:6006
to see tensorboard logs.
To analyze the results:
scripts/analysis.sh
cd scripts
./run_all_analyses.sh <experiment-dir> <apk-dir> <num-agents>
Check the results:
cd <experiment-dir>/tb_otest_logs/analysis
tensorboard --logdir=. --reload_interval 1 --samples_per_plugin "images=0"
Connect to localhost:6006
to see tensorboard logs.
If you use the code or data, please cite the following paper:
@INPROCEEDINGS{9678778,
author={YazdaniBanafsheDaragh, Faraz and Malek, Sam},
title={Deep GUI: Black-box GUI Input Generation with Deep Learning},
booktitle={2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE)},
year={2021}, volume={}, number={}, pages={905-916}, doi={10.1109/ASE51524.2021.9678778}}