[FEATURE] Reduce the size of docker image

[Chan9390]

The docker image size of MobSF is more than 1.7GB. Can we decrease the size of the image ?

A solution could be using a different (lightweight) base image or install only the recommended software to compile and run MobSF.

[Chan9390]

This code in Dockerfile doesnt cleanup anything as removing on one layer will not remove the software from the upper layer.

https://github.com/MobSF/Mobile-Security-Framework-MobSF/blob/9668de85ccb8e3fb355b71be65a601126038ac72/Dockerfile#L74-L87

[Chan9390]

Before I go ahead and optimize, I had a few questions:

1. Was there any reason to use ubuntu as base image ?
2. Why do you install tools like git, libssl-dev, etc and try to uninstall later ? Is it just to compile yara-python ? Wouldn't pip install yara-python work ?

[ajinabraham]

Definitely there is a scope for optimisation. Go for it. I will answer inline.

1. No particular reason, ubuntu was just friendly and well tested. We tried using alpine, but went back from that decision since alpine-python build times are high and we haven't got much size optimisation as expected. https://pythonspeed.com/articles/alpine-docker-python/

2. git is required to install the yara-python fork that works with APKiD. we need to pip install using git protocol. For libssl-dev, it might be a dependency for something else. Try removing it and something might fail.

[Chan9390]

yara-python is also available as a pip package and can be installed as pip3 install yara-python. Any reason to compile it from scratch ?

Are there any tests that will check if MobSF is running fine after few modifications ?

[ajinabraham]

We follow the instructions from https://github.com/rednaga/APKiD#installing As far as I know, we cannot use the PyPI version as we have to install dex enabled yara-python which is not available by default.

A quick test specific to APKiD, yara-python is https://github.com/MobSF/Mobile-Security-Framework-MobSF/blob/master/setup.bat#L47-L49

Otherwise run MobSF tests: https://mobsf.github.io/docs/#/tests after making changes.

[superpoussin22]

you can reduce size using experimental build arg from docker with the --squash parameter and also the --compress option to speed up build ubuntu is also used because we have wkhtmltopdf https://github.com/wkhtmltopdf/wkhtmltopdf

[Chan9390]

I tried the squash experimental docker command and was able to build docker image of size 1.46 GB. The disadvantage is it creates a single layer which needs to be pulled for every new image build. Thats even worse.

Can we have a yara-python fork on MobSF Github org with dex enabled releases ? This would remove the need to install git and other dependencies.

[ajinabraham]

I know that we can host a wheel for that, but not sure if it will be cross platform. will do more testing here.

Even if I just submodule it, it will not save much space because it will still need dependencies for building it. I am thinking if we can release a prebuilt built wheel.

[Chan9390]

I know that we can host a wheel for that, but not sure if it will be cross platform

We will anyway run that inside ubuntu docker image.

I am thinking if we can release a prebuilt built wheel.

Exactly. If we can create a build, we can directly download it during docker build process

[ajinabraham]

The thing is I do not want to create a wheel only for linux/docker and build it locally for others. This is very difficult to maintain and update between operating systems. We give higher priority to maintenance than anything else.

I want to know if we can remove building dependency at user machine because that's a win - win for both user experience and also reducing docker image size.

[ajinabraham]

Once this is merged I think you can do more optimisations on Dockerfile https://github.com/MobSF/Mobile-Security-Framework-MobSF/pull/1377

[ajinabraham]

It's merged to master. @superpoussin22 Can we use ubuntu-minimal as base image ?

[superpoussin22]

@ajinabraham for me LTS starting at 18.04 are minimal "On Dockerhub, the new Ubuntu 18.04 LTS image is now the new Minimal Ubuntu 18.04 image. Launching a Docker instance with docker run ubuntu:18.04 therefore launches a Docker instance with the latest Minimal Ubuntu."

[ajinabraham]

@superpoussin22 Yeah I found out that 20.04 is also minimal image

20.04
linux/amd64

[ajinabraham]

@Chan9390 Our MobSF github repo is about 757 MB on disk and our latest Docker image is about 755.43 MB in DockerHub (but still about 1.7GB on disk). Do you have further optimisations for Dockerfile in mind?

[superpoussin22]

to reduce the size I think we have to options:

1. Get rid of Wkhtmltopdf
2. Remove dynamic analysis from Dockerimage; dockerfile create a file to inform MobSF that it is runing in a container Mobsf does not load the Dynamic analysis menu if the file is present put the dynamic analysis tree in the docignore file

[ajinabraham]

@superpoussin22

1. If only we have cross platform solution that supports html -> pdf generation.
2. Even though it is not officially documented, There are folks who actually make docker to support dynamic analysis with some custom hacks. So we don't want to do that.

[superpoussin22]

I have made the test it won't reduce the size for 2) so not a good idea

[Chan9390]

One more idea is to not copy the .git directory as the git history is not required CMIIW. It will reduce some more space.

EDIT: I found .git in dockerignore. So I think this is already taken care.

[Chan9390]

Adding a --no-install-recommends to apt install command reduced the size of the docker image to 1.65 GB. However, I am not sure if that missed installing any package that MobSF requires at runtime.

[superpoussin22]

without --no-install-recommends I'm at 1.6 but will retry with --no-install-recommends and see if we can reduce the size more

[Chan9390]

@superpoussin22 Did you get time to check it ?

[superpoussin22]

@Chan9390 the PR is ready to merge :)

[ajinabraham]

PR is merged