aquarist-labs / aquarium

Project Aquarium is a SUSE-sponsored open source project aiming at becoming an easy to use, rock solid storage appliance based on Ceph.
https://aquarist-labs.io/
Other
71 stars 23 forks source link

tools: Can not create Vagrant box using builder container #738

Closed votdev closed 2 years ago

votdev commented 2 years ago

I'm not able to build a working Vagrant box using the images/builder/ container. I found a bug which is fixed by #737, but the node does not come up with the image build in the container. Strangely enough, the problem does not occur if the image is built on a desktop system with openSUSE.

Build looks successful:

[ INFO    ]: 17:20:08 | umount FileSystemBtrfs instance
[ INFO    ]: 17:20:08 | Cleaning up VolumeManagerBtrfs instance
[ INFO    ]: 17:20:08 | Cleaning up Disk instance
[ INFO    ]: 17:20:08 | Cleaning up LoopDevice instance
[ INFO    ]: 17:20:08 | Creating vagrant Disk Format
[ INFO    ]: 17:21:13 | Cleaning up DiskFormatVagrantLibVirt instance
[ INFO    ]: 17:21:13 | Result files:
[ INFO    ]: 17:21:13 | --> disk_format_image: /builder/out/aqr-fix_builder-20211103.02/_out/project-aquarium.x86_64-16.0.0.vagrant.libvirt.box
[ INFO    ]: 17:21:13 | --> disk_image: /builder/out/aqr-fix_builder-20211103.02/_out/project-aquarium.x86_64-16.0.0.raw
[ INFO    ]: 17:21:13 | --> image_changes: /builder/out/aqr-fix_builder-20211103.02/_out/project-aquarium.x86_64-16.0.0.changes
[ INFO    ]: 17:21:13 | --> image_packages: /builder/out/aqr-fix_builder-20211103.02/_out/project-aquarium.x86_64-16.0.0.packages
[ INFO    ]: 17:21:13 | --> image_verified: /builder/out/aqr-fix_builder-20211103.02/_out/project-aquarium.x86_64-16.0.0.verified
+ '[' 0 -eq 0 ']'

But deploying the nodes will end up with this: Bildschirmfoto von 2021-11-03 18-41-33

votdev commented 2 years ago

I've contacted @schaefi (a Kiwi deleveloper) and he gave me the following information. The problem is known; it is a Grub thingy. Kiwi uses grub2-mkconfig to configure grub; grub2-mkconfig tries to resolve the UUID of the root device; to do so, UDEV is required, but this is not available in a container environment; because of that grub2-mkconfig uses the device name at the time it is executed which is a loop device. Kiwi has a post processing to fix that issue. It has been introduced with 9.23.21

commit 791adcfcb76753993b5d2c8b2ee66e9dfd8e9880 (origin/fix_use_of_by_label_with
_grub, fix_use_of_by_label_with_grub)
Author: Marcus Schäfer <ms@suse.de>
Date:   Tue Mar 9 12:14:41 2021 +0100

    Fix use of by-label device persistency in grub

    If devicepersistency="by-label" is set in the KIWI description
    it will correctly operate on the fstab values but still uses
    the UUID based setting for root= in the grub config. This commit
    allows to set root=LABEL=... in the grub config in case the
    devicepersistency requested it. In order for this to work this
    commit also had to increase the scope of the grub helper
    method _fix_grub_root_device_reference which is now called in
    any case. This Fixes #1757
votdev commented 2 years ago

I did a new build to check various things. The build/image-root/config.bootoptions file looks OK to me:

quiet systemd.show_status=yes console=ttyS0,115200 console=tty0 net.ifnames=0 \$ignition_firstboot ignition.platform.id=metal root=UUID=2f9e194d-e8da-4532-a16f-97b34817ec3a rw

The UUID of the partition named ROOT is the same: Bildschirmfoto vom 2021-11-04 11-05-29

But what makes me wonder is this: Bildschirmfoto vom 2021-11-04 10-58-54

Who is responsible for creating the dracut-state.sh?

votdev commented 2 years ago

OK, changing the kernel command line to root=/dev/vda3 in the Grub menu after powering the system fixes the issue temporarily.

Bildschirmfoto vom 2021-11-04 11-14-27