search

TuxML / tuxml

The main repository of the TuxML project contains the scripts for building Linux kernel configurations in the large.
Apache License 2.0
4 stars 2 forks source link

Building of the image still need do download packages in its last stage #11

Open mipicard opened 5 years ago

mipicard commented 5 years ago

Since we refactor the management of the docker image, we have drastically reduce the size of the image to download.

For reminding, this is the command run to pre-install all the dependencies : 

BASIC_DEP = All the dependencies

RUN apt-get update && 
    apt-get -qq -y install python3 python3-dev python3-pip python3-setuptools default-libmysqlclient-dev apt-file apt-utils &&
    apt-get install -qq -y --no-install-recommends --download-only  <BASIC_DEP>
RUN echo <BASIC_DEP> > /dependencies.txt

And the command to finally install them :

RUN apt-get install -qq -y --no-install-recommends $(cat /dependencies.txt)

But this method doesn't fully install the dependencies : it only download the selected packages and will download the missing dependencies when running the last command. This behaviour not only add approximately 87 other MB to download, but it also break the image when it's needed packages isn't available anymore. This cause the current prod image to be outdated and to don't work anymore.

So we need to fix it asap.

mipicard commented 5 years ago

I'm thinking about running a python script at the end of the building of basetuxml. This script will do the follow :

mipicard commented 5 years ago

The preceding solution reduce the number of downloaded data from 87 MB to 13 MB in the refactoring branch.

Get:1 http://deb.debian.org/debian stretch/main amd64 flex amd64 2.6.1-1.3 [440 kB]
Get:2 http://deb.debian.org/debian stretch/main amd64 git amd64 1:2.11.0-3+deb9u4 [4167 kB]
Get:3 http://deb.debian.org/debian stretch/main amd64 iptables amd64 1.6.0+snapshot20161117-6 [288 kB]
Get:4 http://deb.debian.org/debian stretch/main amd64 bc amd64 1.06.95-9+b3 [105 kB]
Get:5 http://deb.debian.org/debian stretch/main amd64 bison amd64 2:3.0.4.dfsg-1+b1 [786 kB]
Get:6 http://deb.debian.org/debian stretch/main amd64 btrfs-progs amd64 4.7.3-1 [577 kB]
Get:7 http://deb.debian.org/debian stretch/main amd64 dialog amd64 1.3-20160828-2 [262 kB]
Get:8 http://deb.debian.org/debian stretch/main amd64 gcc-6-plugin-dev amd64 6.3.0-18+deb9u1 [1091 kB]
Get:9 http://deb.debian.org/debian stretch/main amd64 initramfs-tools all 0.130 [66.0 kB]
Get:10 http://deb.debian.org/debian stretch/main amd64 jfsutils amd64 1.1.15-3 [204 kB]
Get:11 http://deb.debian.org/debian stretch/main amd64 libcrypto++6 amd64 5.6.4-7 [829 kB]
Get:12 http://deb.debian.org/debian stretch/main amd64 lzop amd64 1.03-4+b1 [41.8 kB]
Get:13 http://deb.debian.org/debian stretch/main amd64 mcelog amd64 147+dfsg-1 [69.9 kB]
Get:14 http://deb.debian.org/debian stretch/main amd64 moreutils amd64 0.60-1 [71.2 kB]
Get:15 http://deb.debian.org/debian stretch/main amd64 nfs-kernel-server amd64 1:1.3.4-2.1 [124 kB]
Get:16 http://deb.debian.org/debian stretch/main amd64 ppp amd64 2.4.7-1+4 [346 kB]
Get:17 http://deb.debian.org/debian stretch/main amd64 qemu-system amd64 1:2.8+dfsg-6+deb9u5 [65.8 kB]
Get:18 http://deb.debian.org/debian stretch/main amd64 qemu-utils amd64 1:2.8+dfsg-6+deb9u5 [985 kB]
Get:19 http://deb.debian.org/debian stretch/main amd64 qemu amd64 1:2.8+dfsg-6+deb9u5 [151 kB]
Get:20 http://deb.debian.org/debian stretch/main amd64 reiserfsprogs amd64 1:3.6.25-4+b1 [248 kB]
Get:21 http://deb.debian.org/debian stretch/main amd64 squashfs-tools amd64 1:4.3-3+deb9u1 [124 kB]
Get:22 http://deb.debian.org/debian stretch/main amd64 xfsprogs amd64 4.9.0+nmu1 [811 kB]
Get:23 http://deb.debian.org/debian stretch/main amd64 grub-legacy amd64 0.97-72 [293 kB]
Get:24 http://deb.debian.org/debian stretch/main amd64 libdb5.3-dev amd64 5.3.28-12+deb9u1 [760 kB]
Get:25 http://deb.debian.org/debian stretch/main amd64 liblz4-tool amd64 0.0~r131-2+b1 [52.3 kB]
Get:26 http://deb.debian.org/debian stretch/main amd64 pcmciautils amd64 018-8+b1 [30.4 kB]
Get:27 http://deb.debian.org/debian stretch/main amd64 quotatool amd64 1:1.4.12-2+b1 [18.3 kB]
Extracting templates from packages: 100%
Preconfiguring packages ...
Fetched 13.0 MB in 16s (804 kB/s)
mipicard commented 5 years ago

Git removed from the dependencies (we don't use git, so why do we install it?). Fetched 8839 kB in 9s (929 kB/s) So we need to investigate about why flex, iptables, bc, bison, btrfs-progs, dialog, gcc-6-plugin-dev, initramfs-tools, jfsutils, libcrypto++6, lzop, mcelog, moreutils, nfs-kernel-server, ppp, qemu-system, qemu-utils, qemu, reiserfsprogs, squashfs-tools, xfsprogs, grub-legacy, libdb5.3-dev, liblz4-tool, pcmciautils and quotatool aren't downloaded before.

mipicard commented 5 years ago

https://unix.stackexchange.com/questions/408346/how-to-download-package-not-install-it-with-apt-get-command Could help to explore.