search

zbm-dev / zfsbootmenu

ZFS Bootloader for root-on-ZFS systems with support for snapshots and native full disk encryption
https://zfsbootmenu.org
MIT License
836 stars 65 forks source link

Cannot boot Xen kernel: Could not find a free area of memory of 0x3f8001 bytes... #660

Open 0x33taji opened 1 month ago

0x33taji commented 1 month ago

ZFSBootMenu build source

Local build, mkinitcpio

ZFSBootMenu version

ae6167d2cecc620dbd9baa065ab36032687b3eea

Boot environment distribution

Arch Linux

Problem description

I am creating the zfsbootmenu EFI image in a fresh archlinux KVM and then putting the built EFI in /boot/efi/efi/zbm in my debian host. I am trying to boot into xen. I am using a boot selection hook kexec as prescribed in man 7 zfsbootmenu. But I am running into this issue during boot (for debugging purpose I am echo'ing all the important variables and steps),

Trying to launch xen
ZBM_SELECTED_KERNEL=/boot/vmlinuz-6.9.10-amd64
ZBM_SELECTED_INITRAMFS=/boot/initrd.img-6.9.10-amd64
ZBM_SELECTED_BE=zroot/ROOT/debian
ZBM_SELECTED_MOUNTPOINT=/zfsbootmenu/environments/zroot/ROOT/debian/mnt
Trying to kexec /zfsbootmenu/environments/zroot/ROOT/debian/mnt/boot/xen.gz
Could not find a free area of memory of 0x3f8001 bytes...
elf_exec_build_load_relocatable: ELF exec load failed
Nothing has been loaded!
Tearing down USB controller 0000:00:14.0...

After that debian boots up. Expected behavior: Xen should boot and then debian should boot. But Xen is throwing this memory error.

Steps to reproduce

Prepare the fresh archlinux VM from archlinux iso file in KVM (I am using VMManager GUI program)

Disk Size: 10GB
Chipset: Q35
Firmware: OVMF_CODE_4M.fd UEFI
Memory: >512M for dkms compilation
ISO: arch latest iso

Prepare a minimal zfsbootmenu maker vm:

ip addr # note the dhcp ip for ssh
echo 'PermitRootLogin yes' >> /etc/ssh/sshd_config # for ssh root access fast
passwd # set a simple passwd for root then ssh like 'arch'

# << ssh to this machine from host; enables copy pasting on xterm >>
# from https://wiki.archlinux.org/title/Installation_guide
bash # default is sh

loadkeys en
fdisk /dev/vda
g
n (part:1, size:+1G, type:ef)
n (part:2, size: remaining, type:83)
w

mkfs.ext4 /dev/vda2
mkfs.fat -F 32 /dev/vda1

mount /dev/vda2 /mnt
mount --mkdir /dev/vda1 /mnt/boot
pacstrap -K /mnt base linux-lts linux-lts-headers linux-firmware nano vim man networkmanager openssh
genfstab -U /mnt >> /mnt/etc/fstab
arch-chroot /mnt # chroot

passwd # set it to a simple password like 'arch'

hwclock --systohc
echo 'en_US.UTF-8 UTF-8' >> /etc/locale.gen
locale-gen
echo 'LANG=en_US.UTF-8' > /etc/locale.conf
echo 'zbm' > /etc/hostname
echo 'PermitRootLogin yes' >> /etc/ssh/sshd_config
systemctl enable sshd
systemctl enable NetworkManager
echo 'timeout 1' >> /boot/EFI/refind/refind.conf # or change timeout 1 or 2 near the top after some comments
mkinitcpio -P

pacman -S refind
refind-install --usedefault /dev/vda1

cat /etc/fstab
blkid /dev/vda2

# take the uuid from cat /etc/fstab or blkid /dev/vda2 and replace with that
# uuid below
UUID=$(blkid /dev/vda2 | awk '{print $2}' | awk -F '"' '{print $2}')

tee /boot/refind_linux.conf <<EOF
"Boot with standard options"   "root=UUID=$UUID rw initrd=initramfs-linux-lts.img"
"Boot with fallback initramfs" "root=UUID=$UUID rw initrd=initramfs-linux-lts-fallback.img"
"Boot to single-user mode"     "root=UUID=$UUID rw initrd=initramfs-linux-lts.img single"
EOF

exit

umount -R /mnt
reboot

# << ssh to the installed machine >>
pacman -S base-devel git python perl fzf kexec-tools mbuffer

Now, in the minimal fresh Arch VM, having environment as:

# pacman -Qe
base 3-2
base-devel 1-1
fzf 0.54.0-1
git 2.45.2-1
kexec-tools 2.0.28-2
linux-firmware 20240703.e94a2a3b-1
linux-lts 6.6.42-1
linux-lts-headers 6.6.42-1
man-db 2.12.1-1
mbuffer 20240707-1
nano 8.1-1
networkmanager 1.48.6-1
openssh 9.8p1-1
python 3.12.4-1
refind 0.14.2-1
vim 9.1.0611-1
wget 1.24.5-3
zfs-dkms 2.2.4-1
zfs-utils 2.2.4-1

ZFS installation:

# zfs installation ( in arch better to use lts kernel, as the rolling kernel
# cannot compile zfs-dkms as of this time of writing)

cat  <<'EOF' >> /etc/pacman.conf
[archzfs]
Server = https://archzfs.com/$repo/$arch
EOF
curl -L https://archzfs.com/archzfs.gpg |  pacman-key -a -
pacman-key --lsign-key DDF7DB817396A49B2A2723F7403BD972F75D9D76
pacman -Syu
pacman -S zfs-dkms zfs-utils
modprobe zfs

# perl modules required by zfsbootmenu (see project cpanfile)
cpan Sort::Versions YAML::PP boolean

ZFSBootMenu + Xen:

cd /usr/src
git clone --depth=1 https://github.com/zbm-dev/zfsbootmenu
cd zfsbootmenu
make core initcpio

tee /etc/zfsbootmenu/config.yaml <<'EOF'
Global:
  ManageImages: true
  InitCPIO: true
  BootMountPoint: /boot
  DracutConfDir: /etc/zfsbootmenu/dracut.conf.d
  PreHooksDir: /etc/zfsbootmenu/generate-zbm.pre.d
  PostHooksDir: /etc/zfsbootmenu/generate-zbm.post.d
  InitCPIOConfig: /etc/zfsbootmenu/mkinitcpio.conf
Components:
  ImageDir: /boot/EFI/zbm
  Versions: false
  Enabled: false
EFI:
  ImageDir: /boot/EFI/zbm
  Versions: false
  Enabled: true
Kernel:
  CommandLine: ro quiet loglevel=0
EOF

tee /etc/zfsbootmenu/hooks/boot-sel.d/91-xen-boot <<'EOF'
#!/bin/bash

echo "Trying to launch Xen"
echo "ZBM_SELECTED_KERNEL=${ZBM_SELECTED_KERNEL}"
echo "ZBM_SELECTED_INITRAMFS=${ZBM_SELECTED_INITRAMFS}"
echo "ZBM_SELECTED_BE=${ZBM_SELECTED_BE}"
echo "ZBM_SELECTED_MOUNTPOINT=${ZBM_SELECTED_MOUNTPOINT}"

kexec_args=(--module "${ZBM_SELECTED_MOUNTPOINT}${ZBM_SELECTED_KERNEL}" --module "${ZBM_SELECTED_MOUNTPOINT}${ZBM_SELECTED_INITRAMFS}" --command-line "no-real-mode dom0_mem=2048M,max:2048M dom0_max_vcpus=2 dom0_vcpus_pin")

# If a Xen kernel exists in the boot environment, use it
if [[ -e "${ZBM_SELECTED_MOUNTPOINT}/boot/xen.gz" ]]; then
    echo "Trying to kexec ${ZBM_SELECTED_MOUNTPOINT}/boot/xen.gz"
    kexec -a -l "${ZBM_SELECTED_MOUNTPOINT}/boot/xen.gz" --reuse-cmdline "${kexec_args[@]}"
    kexec -e
    exit 0
fi

echo "Could not launch Xen"
# If no Xen kernel or EFI image found, continue with normal boot
EOF
chmod +x /etc/zfsbootmenu/hooks/boot-sel.d/91-xen-boot

generate-zbm
cd /boot/efi/zbm
python3 -m http.server --bind 0.0.0.0 80

# On host debian machine
# cd /boot/efi/efi/zbm
# sudo rm *.efi
# sudo wget http://<archlinux_vm_url>/vmlinuz-linux-lts.EFI
# sudo reboot -ff

Screen during boot (after importing pool):

Trying to launch xen
ZBM_SELECTED_KERNEL=/boot/vmlinuz-6.9.10-amd64
ZBM_SELECTED_INITRAMFS=/boot/initrd.img-6.9.10-amd64
ZBM_SELECTED_BE=zroot/ROOT/debian
ZBM_SELECTED_MOUNTPOINT=/zfsbootmenu/environments/zroot/ROOT/debian/mnt
Trying to kexec /zfsbootmenu/environments/zroot/ROOT/debian/mnt/boot/xen.gz
Could not find a free area of memory of 0x3f8001 bytes...
elf_exec_build_load_relocatable: ELF exec load failed
Nothing has been loaded!
Tearing down USB controller 0000:00:14.0...
zdykstra commented 1 month ago

Double check that you're using the correct kernel image loader for kexecand Xen. Otherwise you'll need to reach out to either Xen or the kexec-tools authors for guidance on how to load and boot your Xen kernel.