search

lxc / distrobuilder

System container image builder for LXC and Incus
https://linuxcontainers.org
Apache License 2.0
562 stars 164 forks source link

[Bug] Distrobuilder fails building VMs on RPM based distros #785

Open rrahl0 opened 9 months ago

rrahl0 commented 9 months ago

Hey, I want to use distrobuilder (git main) on openSUSE Tumbleweed and/or Fedora, and I get everything working (as I need to patch to use mkisofs in comparison to genisoimage (which is debian specific). with the exception of VM building. Even --debug doesn't really give me a good explanation why it doesn't work. Here is the output with the ubuntu.yaml file from the examples dir

# distrobuilder build-incus --vm --debug ubuntu.yaml
[...]
INFO   [2023-10-31T07:20:20+01:00] Running generator                             generator=hostname
INFO   [2023-10-31T07:20:20+01:00] Running generator                             generator=hosts
INFO   [2023-10-31T07:20:20+01:00] Running generator                             generator=remove
INFO   [2023-10-31T07:20:20+01:00] Running generator                             generator=remove
INFO   [2023-10-31T07:20:20+01:00] Running generator                             generator=dump
INFO   [2023-10-31T07:20:20+01:00] Running generator                             generator=copy
INFO   [2023-10-31T07:20:20+01:00] Running generator                             generator=remove
INFO   [2023-10-31T07:20:20+01:00] Running generator                             generator=dump
INFO   [2023-10-31T07:20:20+01:00] Running generator                             generator=fstab
INFO   [2023-10-31T07:20:20+01:00] Running generator                             generator=incus-agent
INFO   [2023-10-31T07:20:20+01:00] Running generator                             generator=dump
INFO   [2023-10-31T07:20:20+01:00] Running generator                             generator=dump
Creating new GPT entries in memory.
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot or after you
run partprobe(8) or kpartx(8)
GPT data structures destroyed! You may now partition the disk using fdisk or
other utilities.
Creating new GPT entries in memory.
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot or after you
run partprobe(8) or kpartx(8)
The operation has completed successfully.
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot or after you
run partprobe(8) or kpartx(8)
The operation has completed successfully.
mke2fs 1.47.0 (5-Feb-2023)
Discarding device blocks: done                            
Creating filesystem with 1022715 4k blocks and 511488 inodes
Filesystem UUID: 4cf9543d-a2ce-47f5-bba7-997cbe8dc09e
Superblock backups stored on blocks: 
    32768, 98304, 163840, 229376, 294912, 819200, 884736

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done 

mkfs.fat 4.2 (2021-01-31)
INFO   [2023-10-31T07:20:29+01:00] Running hooks                                 trigger=post-files
+ TARGET=x86_64
+ uname -m
+ [ x86_64 = aarch64 ]
+ update-grub
Sourcing file `/etc/default/grub'
Sourcing file `/etc/default/grub.d/50-incus.cfg'
Sourcing file `/etc/default/grub.d/init-select.cfg'
Generating grub configuration file ...
grub-probe: error: unknown filesystem.
Found linux image: /boot/vmlinuz-5.4.0-166-generic
Found initrd image: /boot/initrd.img-5.4.0-166-generic
/usr/sbin/grub-probe: error: unknown filesystem.
done
+ grub-install --uefi-secure-boot --target=x86_64-efi --no-nvram --removable
Installing for x86_64-efi platform.
grub-install: error: unknown filesystem.
Error: Failed to run post-files: exit status 1
ERROR  [2023-10-31T07:20:32+01:00] Failed running distrobuilder                  err="Failed to run post-files: exit status 1"
INFO   [2023-10-31T07:20:32+01:00] Removing cache directory

Hope somebody can help me with this.

monstermunchkin commented 9 months ago

Could you please share your ubuntu.yaml?

monstermunchkin commented 9 months ago

Is it only the Ubuntu VM image that fails to build? Have you tried building an Alpine Linux VM image?

rrahl0 commented 9 months ago

Could you please share your ubuntu.yaml?

I use this to test stuff https://github.com/lxc/distrobuilder/blob/main/doc/examples/ubuntu.yaml

Is it only the Ubuntu VM image that fails to build? Have you tried building an Alpine Linux VM image?

No I didn't, as I don't have a yaml file for that (at least for now)

monstermunchkin commented 9 months ago

Could you please share your ubuntu.yaml?

I use this to test stuff https://github.com/lxc/distrobuilder/blob/main/doc/examples/ubuntu.yaml

I suggest you try this one instead https://github.com/lxc/lxc-ci/blob/main/images/ubuntu.yaml. The yaml files in lxc-ci are actively maintained and therefore more up-to-date.

Is it only the Ubuntu VM image that fails to build? Have you tried building an Alpine Linux VM image?

No I didn't, as I don't have a yaml file for that (at least for now)

https://github.com/lxc/lxc-ci/blob/main/images/alpine.yaml

rrahl0 commented 9 months ago

I am sorry for being ignorant, but how can I choose what to build. as both yamls just error out with wrong release. (I assume I have to choose which release to build but can't find anywhere, how to do that)

doh, right after posting this I found out how

rrahl0 commented 9 months ago

so, alpine spits out a different error:

# distrobuilder build-incus --vm --debug alpine.yaml -o image.release=3.16
[...]
INFO   [2023-10-31T08:04:30+01:00] Running generator                             generator=hostname
INFO   [2023-10-31T08:04:30+01:00] Running generator                             generator=hosts
INFO   [2023-10-31T08:04:30+01:00] Running generator                             generator=fstab
INFO   [2023-10-31T08:04:30+01:00] Running generator                             generator=incus-agent
INFO   [2023-10-31T08:04:30+01:00] Running generator                             generator=dump
INFO   [2023-10-31T08:04:30+01:00] Running generator                             generator=dump
INFO   [2023-10-31T08:04:30+01:00] Running generator                             generator=dump
INFO   [2023-10-31T08:04:30+01:00] Running generator                             generator=template
Creating new GPT entries in memory.
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot or after you
run partprobe(8) or kpartx(8)
GPT data structures destroyed! You may now partition the disk using fdisk or
other utilities.
Creating new GPT entries in memory.
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot or after you
run partprobe(8) or kpartx(8)
The operation has completed successfully.
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot or after you
run partprobe(8) or kpartx(8)
The operation has completed successfully.
Error: Failed to mount image: Failed to parse "": strconv.Atoi: parsing "": invalid syntax
ERROR  [2023-10-31T08:04:33+01:00] Failed running distrobuilder                  err="Failed to mount image: Failed to parse \"\": strconv.Atoi: parsing \"\": invalid syntax"
INFO   [2023-10-31T08:04:33+01:00] Removing cache directory

running the command with image.release=edge fixes the bug, and it successfully creates the vm image. so will test the alpine yaml a bit more

for some reason, now I get the same error as ubuntu with 3.16, and edge errors out with above error :(

rrahl0 commented 9 months ago

yeah still the same error with ubuntu:

# distrobuilder build-incus --vm --debug ubuntu -o image.release=jammy
[...]
INFO   [2023-10-31T08:16:14+01:00] Running generator                             generator=hostname
INFO   [2023-10-31T08:16:14+01:00] Running generator                             generator=hosts
INFO   [2023-10-31T08:16:14+01:00] Running generator                             generator=remove
INFO   [2023-10-31T08:16:14+01:00] Running generator                             generator=remove
INFO   [2023-10-31T08:16:14+01:00] Running generator                             generator=dump
INFO   [2023-10-31T08:16:14+01:00] Running generator                             generator=remove
INFO   [2023-10-31T08:16:14+01:00] Running generator                             generator=dump
INFO   [2023-10-31T08:16:14+01:00] Running generator                             generator=fstab
INFO   [2023-10-31T08:16:14+01:00] Running generator                             generator=incus-agent
INFO   [2023-10-31T08:16:14+01:00] Running generator                             generator=dump
INFO   [2023-10-31T08:16:14+01:00] Running generator                             generator=dump
Creating new GPT entries in memory.
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot or after you
run partprobe(8) or kpartx(8)
GPT data structures destroyed! You may now partition the disk using fdisk or
other utilities.
Creating new GPT entries in memory.
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot or after you
run partprobe(8) or kpartx(8)
The operation has completed successfully.
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot or after you
run partprobe(8) or kpartx(8)
The operation has completed successfully.
mke2fs 1.47.0 (5-Feb-2023)
Discarding device blocks: done                            
Creating filesystem with 1022715 4k blocks and 511488 inodes
Filesystem UUID: ce307809-5268-4141-9c96-451ad3ee004a
Superblock backups stored on blocks: 
    32768, 98304, 163840, 229376, 294912, 819200, 884736

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done 

mkfs.fat 4.2 (2021-01-31)
INFO   [2023-10-31T08:16:20+01:00] Running hooks                                 trigger=post-files
+ TARGET=x86_64
+ uname -m
+ [ x86_64 = aarch64 ]
+ update-grub
Sourcing file `/etc/default/grub'
Sourcing file `/etc/default/grub.d/50-incus.cfg'
Sourcing file `/etc/default/grub.d/init-select.cfg'
Generating grub configuration file ...
grub-probe: error: unknown filesystem.
Found linux image: /boot/vmlinuz-5.15.0-88-generic
Found initrd image: /boot/initrd.img-5.15.0-88-generic
/usr/sbin/grub-probe: error: unknown filesystem.
done
+ grub-install --uefi-secure-boot --target=x86_64-efi --no-nvram --removable
Installing for x86_64-efi platform.
grub-install: error: unknown filesystem.
Error: Failed to run post-files: exit status 1
ERROR  [2023-10-31T08:16:22+01:00] Failed running distrobuilder                  err="Failed to run post-files: exit status 1"
INFO   [2023-10-31T08:16:22+01:00] Removing cache directory
monstermunchkin commented 9 months ago

Looking at grub-install --help, there's the option --skip-fs-probe. You can try that, perhaps it'll fix the issue.

rrahl0 commented 9 months ago

Looking at grub-install --help, there's the option --skip-fs-probe. You can try that, perhaps it'll fix the issue.

so this does work for alpine, but not ubuntu. which kinda weirds me out, as I thought the whole grub part is getting executed inside the vm, which would mean it shouldn't even come up as an issue (as it works on a 23.10 ubuntu vm.)

here is the part which is relevant:

+ grub-install --uefi-secure-boot --target=x86_64-efi --no-nvram --removable --skip-fs-probe
Installing for x86_64-efi platform.
grub-install: error: unknown filesystem.
Error: Failed to run post-files: exit status 1
ERROR  [2023-10-31T09:09:00+01:00] Failed running distrobuilder                  err="Failed to run post-files: exit status 1"
INFO   [2023-10-31T09:09:00+01:00] Removing cache directory
monstermunchkin commented 9 months ago

so this does work for alpine, but not ubuntu. which kinda weirds me out, as I thought the whole grub part is getting executed inside the vm, which would mean it shouldn't even come up as an issue (as it works on a 23.10 ubuntu vm.)

This is not executed in a VM, but in a chroot. Is it perhaps your host that doesn't support secure boot or UEFI, and therefore grub's not happy?

rrahl0 commented 9 months ago

@monstermunchkin the vm is setup with UEFI+SB with only MS keys. in the case of tumbleweed, I also enabled trusted boot