Profile and reduce host OS image size

willnewton commented 7 years ago

Profile the on-disk size of resinOS and find areas where it can be reduced.

shaunmulligan commented 7 years ago

maybe look at some of the useless kernel modules we add by default, like PS/2 mouse and joystick drivers :P

willnewton commented 7 years ago

This is the list of installed packages. Total size is 218334kB, of which:

docker is 27342kB
linux-firmware-* is 50756kB
kernel-module* is 44091kB
The rest is 96137kB

Of the rest a large part of that is pulled in by systemd/networkmanager I think.

A good place to start would be slimming down firmwares for devices that do not need them, e.g. the Intel Wifi firmwares (40MB of them) don't make a whole lot of sense on non-Intel platforms.

Other things that stand out from the list that could be removed:

kernel-module-ocfs2 (1230kB)
lvm2 (4290kB + dependencies) do we need LVM? at least on rpi type boards I would think not
dhcp-client (2078kB) we probably have a DHCP client in systemd and busybox, do we need this one?
systemd-analyze (1223kB) should be restricted to dev images?
libgnutls30 (1198kB + dependencies) for some reason yocto likes to build things (e.g. wpa_supplicant, curl) with gnutls, would be good to use openssl instead if possible
bash (806kB) it is reasonably large but we have quite a lot of bash scripts...
kernel-module-drbd (487kB)
kernel-module-gfs2 (412kB)
Other filesystems, e.g. cifs, nfs (may be useful for NAS applications), reiserfs, ntfs, jfs, hfsplus, udf all in the 100s of kB range

installed-package-sizes.txt

willnewton commented 7 years ago

It looks like the lvm2 dependency is dropped by the hostapps PR:

https://github.com/resin-os/meta-resin/pull/805/commits/f5c783df899172d9e904818835a6da575d6631ab

willnewton commented 7 years ago

Opened a PR to reduce kernel size here:

https://github.com/resin-os/meta-resin/pull/813

imrehg commented 7 years ago

@willnewton would the file systems make sense to keep since people can use external storage, etc? (and they did use, e.g. external USB stick and such) Or if those are removed from here, would people be able to add them back easily in the container?

willnewton commented 7 years ago

Opened a PR for systemd-analyze here:

https://github.com/resin-os/meta-resin/pull/814

willnewton commented 7 years ago

@imrehg the fileystems are mostly for servers or osbolete:

ocfs2 - cluster filesystem
gfs2 - cluster filesystem
jfs - obsolete AIX filesystem
hfs - Obsolete Apple filesystem
hfs+ - More modern Apple filesystem but still rare for removable disks
ntfs - Partial read-only support for Windows NTFS
udf - DVD filesystem support
reiserfs - No longer developed AFAIK

I left in NFS and CIFS as these seem legitimately useful, we also keep XFS.

alexgg commented 4 years ago

The comments above have size optimized the following:

Kernel configuration
Running services
Firmware files

Although the effort above can be repeated, it is unlikely that we would find any big space saver left. The analysis for v2.48 is summarized in:

284M    /build/balena-intel/build/tmp/work/genericx86_64-poky-linux/resin-image/1.0-r0/rootfs/

36M     /lib/modules
61M     /lib/firmware
45M     /usr/lib
34.5M   /usr/bin/balena-engine
12.2M   /boot/bzImage
7.2M    /boot/init
6.4M    /resin-boot
5M      /usr/bin/os-config
3.1M    /usr/lib/rust/libstd.so
2.8M    /usr/sbin/NetworkManager
1.5M    /usr/sbin/ModemManager
1.4M    /lib/systemd/systemd
1M      /bin/bash.bash

218M, 67M unaccounted (< 1M objects)

On v2.51 with the update of balena-engine the filesystem has gone above it's threshold again. Looking at the above list the following actions can be considered:

Enable kernel compression

Saves 23M
Works on the supported Alpine and Fedora images
Only works from Debian Buster and Ubuntu Cosmic

Enable firmware compression

Requires a kernel >= v5.3

Enable balena-engine compression

Saves 33M
Increases boot time by 5s in Intel NUC

Enable os-config compression

Saves 4M
Increases boot time in less than 1s

Removing unneeded files from /boot directory

Saves around 600K for grub based devices, 400K for u-boot based devices

Also, the inactive partition is 300M, but a 5% is reserved by default (~15M) so only a hostapp of 285M can be updated. As the root filesystem is read-only and nothing read or writes to it, there is no reason not to remove this reserve and use the whole available space, which is 300M minus the 8M journal size.

alexgg commented 2 years ago

Updating on this, according to https://wiki.debian.org/LTS Strech has gone EOL on the 30th June 2022. And according to https://ubuntu.com/about/release-cycle 18.04 LTS (Bionic Beaver) is now under ESM (Extended Security Maintenance). It's a good moment to explore adding module compression as default and checking that Stretch and Bionic can still use non-compressed kernel modules.

jellyfish-bot commented 2 years ago

[alexgg] This has attached https://jel.ly.fish/6e6207ca-a304-4b94-88fe-0cb6ddd40b2d

alexgg commented 1 year ago

Bullseye supports module compression:

root@87d09b8ecf24:/# cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
root@87d09b8ecf24:/# modprobe -V
kmod version 28
-ZSTD +XZ -ZLIB +LIBCRYPTO -EXPERIMENTAL

balena-os / meta-balena

Profile and reduce host OS image size #810