Alpine Linux (Virt) packer script

[Max13]

Packer tested only on macOS Monterey, takes 1m12s with hvf accelerator.

I hope the timing will be OK on Linux too.

[grossmj]

Thanks a lot for this

[Max13]

@grossmj You're welcome, and as @b-ehlers pointed our, it will be easier for me to update the whole thing afterward.

BTW, does it mean that, at some point, the image will be available on GNS3 server directly ?

[grossmj]

BTW, does it mean that, at some point, the image will be available on GNS3 server directly ?

I have tested on Linux and uploaded the Qemu image on our Sourceforge account. The MD5 checksum has changed. Please can you check?

https://github.com/GNS3/gns3-registry/pull/693

Thanks.

[Max13]

The MD5 checksum has changed. Please can you check?

Every time you generate the image, the checksum changes. I can't explain exactly why. I did some dd if=img_file.img bs=1M count=1 | md5 on 2 different images (generated one after the other) but I get this:

dd if=output-qemu/alpine-virt-3.16-1.img bs=1M count=1 skip=4 2>/dev/null | md5; dd if=output-qemu/alpine-virt-3.16-2.img bs=1M count=1 skip=4 2>/dev/null | md5
b6d81b360a5672d80c27430f39153e2c
b6d81b360a5672d80c27430f39153e2c

So I suppose, as the first partition is at sector 2048, and the MBR isn't that large, it could mean random data between MBR and sector 2048.

If it's true, the only way to get exactly the same hash between each generations, is to write 0's between sector 0 and 2048 before starting. Not ideal I think.

[b-ehlers]

Using the MD5 checksum for comparing different builds is pointless, as every build has a different MD5. The main reason are different timestamps of directories and configuration files. Other reasons are documented here: https://reproducible-builds.org/docs/system-images/

Another way is to mount the images and compare the files:

$ mkdir max13 grossmj
$ /sbin/fdisk -l alpine-virt-3.16.img.max13
Disk alpine-virt-3.16.img.max13: 92 MiB, 96468992 bytes, 188416 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xbc8c65d6

Device                       Boot Start    End Sectors Size Id Type
alpine-virt-3.16.img.max13p1 *     2048 188415  186368  91M 83 Linux
$ sudo mount -r -o loop,offset=$((2048*512)) alpine-virt-3.16.img.max13 max13
$ sudo mount -r -o loop,offset=$((2048*512)) alpine-virt-3.16.img.grossmj grossmj
$ sudo rsync -ancO --delete --info=name1 max13/ grossmj/
deleting boot/extlinux.conf.bkp
deleting etc/update-extlinux.d/
deleting etc/inittab.bkp
deleting lib/modules/5.15.71-0-virt/kernel/virt/lib/irqbypass.ko.gz
deleting lib/modules/5.15.71-0-virt/kernel/virt/lib/
deleting lib/modules/5.15.71-0-virt/kernel/virt/
deleting lib/modules/5.15.71-0-virt/kernel/security/keys/encrypted-keys/encrypted-keys.ko.gz
deleting lib/modules/5.15.71-0-virt/kernel/security/keys/encrypted-keys/
deleting lib/modules/5.15.71-0-virt/kernel/security/keys/
deleting lib/modules/5.15.71-0-virt/kernel/security/
.
.
In total 2508 changed files

Why this big number of changed files? I booted both images and created a list of installed packages with apk list -I | sort, then saved and compared them:

$ diff -u pkg.max13 pkg.grossmj | grep "^[+-]"
--- pkg.max13   2022-10-05 04:30:55.095635977 +0200
+++ pkg.grossmj 2022-10-05 04:30:11.732348237 +0200
+blkid-2.38-r1 x86_64 {util-linux} (GPL-3.0-or-later AND GPL-2.0-or-later AND GP]
-linux-virt-5.15.70-r0 x86_64 {linux-lts} (GPL-2.0) [installed]
+linux-virt-5.15.71-r0 x86_64 {linux-lts} (GPL-2.0) [installed]
+mtools-4.0.39-r0 x86_64 {mtools} (GPL-3.0-or-later) [installed]
+syslinux-6.04_pre1-r10 x86_64 {syslinux} (GPL-2.0-or-later) [installed]

The image from @grossmj uses an updated linux-virt package. Furthermore it had additionally installed syslinux (plus the dependencies blkid and mtools), most likely because the image from @Max13 was created manually without installing syslinux.

When looking at the list of changed files, the vast majority is indeed caused by the difference of installed packages. But there are some files left, that need some attention. So even these small changes need several hours to verify the differences, that is almost impossible to do. So my suggestion is, that @Max13 boots and tests the image of @grossmj and when everything looks good, he approves it.

[grossmj]

@b-ehlers Thanks for this, I especially like the method to compare the content of the 2 images.

@Max13 As suggested by @b-ehlers, please can you confirm that the image work as intended on your side? I will merge my PR once you have confirmed. Thanks.

[b-ehlers]

Just build my own version of alpine-virt-3.16.img and compared that with the image from @grossmj.

$ sudo mount -r -o loop,offset=$((2048*512)) alpine-virt-3.16.img.grossmj grossmj
$ sudo mount -r -o loop,offset=$((2048*512)) alpine-virt-3.16.img.behlers behlers
$ sudo rsync -ancO --delete --info=name1 grossmj/ behlers/
boot/extlinux.conf
boot/extlinux.conf.bkp
boot/initramfs-virt
etc/fstab
etc/update-extlinux.conf
lib/apk/db/scripts.tar

Not surprising the differences are quite small now, they are mainly caused by different UUIDs of the root filesystem. But still checking the differences is a manual process.

[Max13]

Please can you check?

Sorry, silly me didn't understand you were talking about checking the image... I checked it manually in GNS3 with NAT, everything is fine.

I will need to send a minor fix later about fsck not present (by choice) while init executes fsck. It's not delaying the VM, it's not critical, how would be the best way to do that? A new version? A fix directly on the current version? Wait for the next Alpine minor release?

[grossmj]

Sorry, silly me didn't understand you were talking about checking the image... I checked it manually in GNS3 with NAT, everything is fine.

No worries, I merged my PR now.

I will need to send a minor fix later about fsck not present (by choice) while init executes fsck. It's not delaying the VM, it's not critical, how would be the best way to do that? A new version? A fix directly on the current version? Wait for the next Alpine minor release?

I would create a new version or wait for the next Alpine minor release if this isn't urgent.

[Max13]

@grossmj Good, I'll wait.

BTW, I was working with my students yesterday (October 6th), they all had GNS3 started and even when they updated the GNS3 registry (with the button, while looking at available appliances) alpine linux virt wasn't shown even my own version downloadable on Google Drive. Did you disable it while waiting for my confirmation, or something like that ?

I'm just trying to understand if I missed something during the install process, strangely it was shown on my Mac (maybe because I had the first version, but the students didn't)

[b-ehlers]

I have no issues adding the alpine-linux-virt appliance to a fresh GNS3 installation:

https://user-images.githubusercontent.com/49040325/194657245-6c2d376c-99e2-4c4a-bc9e-24cba7a140d5.mp4

When starting the "New template" wizard, the alpine-linux-virt appliance is not available. After an update from the online registry it appears and can be selected. During the installation of the appliance the image needs to be downloaded, that takes quite a while in my environment. When that finally succeeds the installation can be finished.

The main drawback of the update of the online registry is, that it wants to save the updated appliances to a subdirectory within the GNS3 installation folder. If that isn't writable by the user, the update fails, see also https://github.com/GNS3/gns3-server/issues/2151