This project encompasses three Dockerfiles, which are:
/submodules
folder.To complete the configuration of the repository, it will be necessary to clone the following repository inside the path /submodules
In the TestCasesScripts project, there exist configuration variables (/submodules/TestCasesScripts/config/methods_config.yml
) that can be modified to set your preferences. These variables include the export format (xlxs or csv), the use of semgrep during test case execution and the database name. It is important to specify the Android version of the device you are going to analyze (you must choose between 13 or 14 in the androidVersion
parameter), depending on the version you choose, the script will consider a different set of permissions. Before building the Docker containers, it is essential to adjust these variables to your desired values.
Once everything is configured, it's time to build the dockers by executing:
sudo docker build -t ubuntu_base:latest -f dockerfile_ubuntu_base .;
sudo docker build -t android-scoring-testcases:latest -f dockerfile_testcases .;
sudo docker build -t android-scoring-extractor:latest -f dockerfile_extractor .;
Then, analysis can be initiated. To proceed, store the image of the application or set of applications and execute the file run.sh using the following command:
sudo ./run.sh True
sudo ./run.sh
Dockers can be run in two different ways: through the image of a device or through a set of applications of your choice.
To scan the image from a device, it is necessary to store the image in the /data/images/
folder. Within the /data/images/
directory, create a subfolder named after the vendor for each image. For example, /data/images/<VENDOR_NAME>/
.
When inserting the image of a device into the /<VENDOR_NAME>
folder, it must be a compressed file, such as .zip, .rar or .tar.gz.
It is also possible to scan a set of applications without the need of a device image. This can be achieved by storing all the applications (apk files) in the following path: /data/ramdisk/apks/
This Python project is designed to perform static analysis of Android Apps to cover 10 MASA test cases (CODE-1, CODE-2, CRYPTO-1, CRYPTO-3, NETWORK-1, NETWORK-2, NETWORK-3, PLATFORM-2, PLATFORM-3 and STORAGE-2). The main script extracts details such as the APK name, version, urls and permissions requested by the application, based on these parameters and the execution of the test cases, it provides a value that indicates the risk score of a set of applications.
The results of the analysis are stored both in a MySQL database distributed in different tables and in an Excel file, which allows easy access and visualization of the data collected. In this excel you can see four tabs which are:
The first step is to install the necessary packages using the requirements.txt file by executing the following command:
pip install -r requirements.txt
In order to register the data in the database, it will be necessary to set up a mysql database following these steps:
sudo apt install mysql-server
sudo systemctl start mysql
mysql -u root -p
masa_script
, as in settings.py , and grant it privileges. One can
also choose other credentials and update the file settings.pymysql> CREATE USER masa_script@localhost IDENTIFIED BY 'MASA123';
mysql> GRANT ALL PRIVILEGES ON *.* TO masa_script@localhost;
mysql> FLUSH PRIVILEGES;
mysql> exit;
Once the necessary packages have been installed, a folder called apks has to be created, where the apks to be analysed will be stored. To store applications in the apks folder, applications will have to be separated by folders as follows:
/scripts-masa
: Main project directory.
/apks
: Directory that will contain all apks./folder_1
: Subdirectory containing apk_1.
app_1.apk
/folder_2
: Subdirectory containing apk_2.
app_2.apk
/folder_3
: Subdirectory containing apk_3.
app_3.apk
Once the database has been created, it will be necessary to execute the following command to create all the tables and structure of the database:
python3 -c "from db.database_utils import first_execution; first_execution()"
With the applications already stored in /apks
and the database tables created, it is now possible to run the script using the following command:
./automate_apps_updated
As a result of the analysis the data obtained will be exported, there are two options CSV or XLXS, to choose the format you must access the configuration file located in the path /config/methods_config.yml
and give value (True/False) to the variable export_csv. The exported data can be found in the folder /Results/YYYYYYYYYYMMdd/HHmmss.
In case you want to delete all the data in the database, you will need to execute the following command:
python3 -c "from db.database_utils import clear_database; clear_database()"
TestSSL_URL:
This table stores the URLs that have already been scanned in NETWORK-2 test case and the result.
Findings:
This table stores all the matches found in the test cases. It stores the hash of the application and the app_name, as well as the category and ID of the test case and the path where the match is located, along with the line number in the file. In some test cases where multiline match is required, it is not possible to get the line number of the match.
Report:
This table stores the result of the test cases (PASS, FAIL or Need Review), information about the scan and information of the application. The application information stored is the hash, app_name and version_name to identify it. Regarding the scan information, it stores if semgrep option is enabled or not and the version of the script used in the analysis.
Total_Fail_Counts:
This table stores the hash of the application along with the number of matches found for each test case.
Logging:
This table stores information about events that occurred during the execution of the script. The hash of the application and the time of the event is also stored, as well as the error message.
Permissions:
This table stores the Android permissions required by the application.
Executions:
This table stores the identifier of the analysis with the timestamp.
To understand the result of the Risk Score, how its value is obtained, what parameters are taken into account and what formula is used, read this paper.
What is the Shared User Id (SUID)? The Shared User Id is a tag found in the AndroidManifest.xml and influences the formula value.
In order to run the script and get the results, you will need to use additional tools found in the /tools
directory, which are:
In case you want to add new test cases, you have to follow the next steps:
/config/methods_config.yml
file and add the name of the new test case to the corresponding category (CODE, CRYPTO, NETWORK...) or create a new one if it does not exist./tests/section/test_name.py
where section will be the name of the category and test_name the name of the test case to be added and copy the new test case./tests/section/test_name.py
implement the test case code in a method called check(), the parameters that the check() method of all the test cases will receive will always be the same for all of them, which are defined in the file /utils/auxiliary_functions.py
in the check_app method in the variable all_params. So if you need to add any additional variable you would have to add it in all_params and modify the headers of all the check() methods of each test case adding this/these new variable(s).If you want the risk score to take into account the newly developed test case, you would have to add to the database tables (Report and Total_Fail_Counts) a new column with the name of the test case in capital letters (in case of adding new test cases to the database, the unit test that checks the formula would become FAIL).
Once these steps have been followed, the execution of the script will also analyse the newly added test case. To check that no modifications have been made to the results of the rest of the test cases and that they are kept originally, we will check the results of the unit tests.
To perform the execution of the unit tests it is necessary to follow several steps to have the environment correctly set up, since we need to specify the name of the database by means of a config variable (database) and the creation of a database for testing (automated_MASA_TESTING).
python3 -c "from db.database_utils import first_execution; first_execution('automated_MASA_TESTING')"
database
variable to automated_MASA_TESTING
./automated_apps_updated True
With this command we will run an analysis of the applications but we will use the variables, directories and databases destined for unit tests.
pytest
python3 -c "from db.database_utils import clear_database; clear_database('automated_MASA_TESTING')"
If you want to re-launch the script with the normal behaviour it will be necessary to revert the changes, to do this we will have to change the value of the variable of the configuration file database
to automated_MASA
and launch the script with the command
./automated_apps_updated
If the application does not contain source code directly in the package you are decompiling, jadx will not be able to create a /sources folder because there is simply no source code to extract.
This project currently uses a two-branch strategy to manage development and releases. Below is an explanation of the purpose of each branch and guidelines for contributing to the project.
Main Branch (main)
Development Branch (dev)
Create a feature branch:
git checkout dev
git checkout -b feature/your-feature-name
Develop your feature:
Push your feature branch:
git push origin feature/your-feature-name
Create a pull request:
Review and merge: