lima-vm / lima

Linux virtual machines, with a focus on running containers
https://lima-vm.io/
Apache License 2.0
15.48k stars 608 forks source link

Debian-slim as Alpine alternative concerning disk footprint #1614

Open pr-ju opened 1 year ago

pr-ju commented 1 year ago

Description

Since SSD space is often an issue on macs but Alpine uses a different c library, which can result in poor perforce when running a python (https://pythonspeed.com/articles/alpine-docker-python/).

Therefore, I would recommend supporting Debian-slim as small alternative as template for a VM with less disk footprint. . With Debian-slim the footprint is ca. 82 MB + kernel resources, which is larger than Alpine but still okay. Maybe Ubuntu minimal is another option here, but Debian-slim is smaller.

Sadly, Debian-slim is not available as official image. Debian cloud images (qcow2) It is only available inside a container registry/repository as with Docker. For managing that there is actually a repository

https://github.com/debuerreotype/docker-debian-artifacts/blob/4648ad5c6feb1318b8a6c31502ebde8bb18ebb5e/bullseye/slim/

Of course I could use that as containers inside a VM that has containerd + nerdctl, but often for development purposes three or four different small VMs (one with python, one with node, etc.) is all that is needed for different projects, which are only running when I'm actually working on them. Even with 4 Debian-slim VMs in lima it would be way less disk space than having a full flavored Ubuntu-VM with 4 containers in it ( > 900MB), let alone running an additional layer of abstraction.
In that fashion I did this with multiple alpine VMs, but ran across those python performance issues.

Best

P.R.

afbjorklund commented 1 year ago

As previously discussed:

I think the C library of the VM is irrelevant to performance, as each container will have its own libc in the image. It could still be worthwhile to discuss base image sizes, but so far the added maintenance has not made up for it...

56M https://dl-cdn.alpinelinux.org/alpine/v3.18/releases/x86_64/alpine-virt-3.18.0-x86_64.iso
67M https://github.com/lima-vm/alpine-lima/releases/download/v0.2.30/alpine-lima-std-3.18.0-x86_64.iso
319M    https://cloud.debian.org/images/cloud/bullseye/20230124-1270/debian-11-generic-amd64-20230124-1270.qcow2
779M    https://cloud-images.ubuntu.com/releases/23.04/release-20230502/ubuntu-23.04-server-cloudimg-amd64.img

That the lima-adapted version of the image is larger than the original, is mostly because it bundles some required packages. For the debian and ubuntu images, those packages are downloaded during installation and installed.

afbjorklund commented 1 year ago

One finding is that we could save some extra disk space, going from "generic" to "genericcloud" (virtual only)

That way, we don't have to include drivers for physical hardware, but only for virtual hardware (like alpine virt)

https://cloud.debian.org/images/cloud/

generic: Should run in any environment using cloud-init, for e.g. OpenStack, DigitalOcean and also on bare metal.

318M

genericcloud: Similar to generic. Should run in any virtualised environment. Is smaller than generic by excluding drivers for physical hardware.

246M (-22.6%)

EDIT:


Haven't tried running the Docker "slim" on it:

debuerreotype-slimify

https://github.com/debuerreotype/debuerreotype

I think it is mostly about removing documentation.

31M /usr/share/doc
888K    /usr/share/info
352K    /usr/share/lintian/overrides
53M /usr/share/locale
13M /usr/share/man
97M total
afbjorklund commented 1 year ago

Most of the extra runtime disk usage actually comes from the apt lists, and not from the deb packages:

130M /var/lib/apt/lists

100K    /var/cache/apt/archives/dbus-user-session_1.12.24-0+deb11u1_amd64.deb
36K /var/cache/apt/archives/fuse3_3.10.3-2_amd64.deb
84K /var/cache/apt/archives/libfuse3-3_3.10.3-2_amd64.deb
0   /var/cache/apt/archives/lock
4.0K    /var/cache/apt/archives/partial
48K /var/cache/apt/archives/sshfs_3.7.1+repack-2_amd64.deb
220K    /var/cache/apt/archives/uidmap_1%3a4.8.1-1_amd64.deb
492K    total
afbjorklund commented 1 year ago

One difference between ubuntu and debian, would be the older kernel being used. This has pros and cons.

ubuntu: 6.2.0

debian: 5.10.0

That is, there might be some bugs in the latest and greatest - similar to using "ubuntu" vs using "ubuntu-lts"

pr-ju commented 1 year ago

Thank you very much, Anders!