reduce size of docker images - review installed software

[SSE4]

currently provided docker images are pretty heavyweight. for instance, conanio/gcc8 currently takes 1,03Gb. as docker images are pulled on each build, it introduces additional delay on build job, which scales linearly based on number of travis jobs. I think it makes a priority to keep our docker images as small as possible. I have reviewed conanio/gcc8 image a bit (via ncdu program - sudo ncdu / command) and spot the following:

• valgrind - /usr/lib/valgrind takes 63,7 Mb. I believe valgrind is usually used only for memory profiling, is it really necessary to include it into build images?
• vim - /usr/share/vim takes 31 Mb. also, do we need text editor, especially such a heavy-weight as part of build images?
• cmake - obviously, CMake itself is required, no doubts here. however, /usr/bin/cmake-gui is 24,7 Mb, plus /doc/cmake is 24,2 Mb, plus /usr/share/cmake-3.12/Help is 6,3 Mb. I don't think we need GUI and documentation for CI builds, right?
• python - first off, we install python 3.7, but there is python 3.6 installation (seems like it's dependency of some other package) - not sure if we can avoid having two pythons? second, /opt/pyenv/versions/3.7.1/lib/python3.7/test/__pycache__ takes 32Mb - I doubt we need it at all. there are few other __pycache__ directories around.

from approximate calculation, removing the above may easily save ~200Mb per image (which is 20% of size).

NOTE: docker pull time on travis is currently around 30-40 seconds.

/cc @lasote @uilianries @danimtb @jgsogo @memsharded

[uilianries]

Amazing investigation! I totally agree with you!

valgrind - /usr/lib/valgrind takes 63,7 Mb. I believe valgrind is usually used only for memory profiling, is it really necessary to include it into build images?

Maybe we could provide valgrind_installer package instead.

vim - /usr/share/vim takes 31 Mb. also, do we need text editor, especially such a heavy-weight as part of build images?

No. Our images were created to build packages only. Also all images have vi installed.

cmake - obviously, CMake itself is required, no doubts here. however, /usr/bin/cmake-gui is 24,7 Mb, plus /doc/cmake is 24,2 Mb, plus /usr/share/cmake-3.12/Help is 6,3 Mb. I don't think we need GUI and documentation for CI builds, right?

Correct, we should remove GUI and doc.

python - first off, we install python 3.7, but there is python 3.6 installation (seems like it's dependency of some other package) - not sure if we can avoid having two pythons? second, /opt/pyenv/versions/3.7.1/lib/python3.7/test/pycache takes 32Mb - I doubt we need it at all. there are few other pycache directories around.

Where did you detect two python versions? I know about python2 and python3 in same image. All extra packages could be removed I think. Few months ago I worked with embedded python and many modules were removed with no problem.

Thanks for bringing us this discussion.

[SSE4]

for valgrind - users always can install it if needed, either via SystemPackageTool, or via docker_entry_script (CPT script) or just make images inherited from ours, if needed.

for python 3.6 + python 3.7 - checkout out the build log, for instance: https://api.travis-ci.org/v3/job/468091286/log.txt it has the following entries:

Selecting previously unselected package libpython3.6-minimal:amd64.
Preparing to unpack .../020-libpython3.6-minimal_3.6.7-1~18.04_amd64.deb ...
Unpacking libpython3.6-minimal:amd64 (3.6.7-1~18.04) ...

I think it is getting installed as a dependency of some another package, but I didn't check which one.

[uilianries]

for valgrind - users always can install it if needed, either via SystemPackageTool, or via docker_entry_script (CPT script) or just make images inherited from ours, if needed.

Sure, and we could use Valgrind as example in CPT readme.

I think it is getting installed as a dependency of some another package, but I didn't check which one.

You are right, seems like be transitive dependency from apt packages. It's hard to be solved, because we will need to detect which package is and try to install without python.

[SSE4]

seems to be dependency of vim:

root@838d679ecacc:/# apt-rdepends vim | grep python
Reading package lists... Done
Building dependency tree       
Reading state information... Done
  Depends: libpython3.6 (>= 3.6.4~rc1)
libpython3.6
  Depends: libpython3.6-stdlib (= 3.6.7-1~18.04)
libpython3.6-stdlib
  Depends: libpython3.6-minimal (= 3.6.7-1~18.04)
libpython3.6-minimal

so, excluding vim is killing two birds in one stone.

[uilianries]

great! :clap:

[memsharded]

There is one consideration here, is that we sometimes use these images for testing and debugging too, don`t you? If we remove vim from the images, should we install it every time we use them? Agree with cmake extra things, trying to keep down to one python version and even removing valgrind.

[uilianries]

for vim case you could mount your container volume and preserve any change in your host:

docker run -ti -v ${PWD}:/home/conan/project conanio/gcc8

Now you are able to edit the code in your host.

I have a bunch of alias to start my docker commands:

alias docker-conan-gcc8='docker run --rm -ti -v ${PWD}:/home/conan/project conanio/gcc8 bash -c "pip install bincrafters_package_tools; conan user; cd project; /bin/bash"'

I would say testing and debugging are extra requirements for our images, on Linux my guess is 5% of cases I need to debug something.

[SSE4]

@memsharded current images are not that useful for debugging, as they don't contain gdb. but both gdb and vim may be easily installed (even via alias script, as @uilianries mentioned).