VMs no longer boot with upgrade from 7.4 to 8

[jarrodthebobo]

After following the documented steps to upgrade to the newest proxmox version, my vms now no longer boot. I also cannot get into the uefi menu. It appears that it just completly hangs.

It looks like the issue we had here https://github.com/jiangcuo/Proxmox-Arm64/issues/20 may have returned?

Im on the rock 5b

[titogrima]

Hi!!

I have same error and I can solved it with downgrade pve-edk2-firmware same that #20 and create a symbolic link in this path /usr/share/pve-edk2-firmware/

In theory it was solved in the version of the package that is installed with the PVE 8 in this ticket https://bugzilla.proxmox.com/show_bug.cgi?id=4696 open by @jiangcuo but don't work

Im on orange pi 5

[jarrodthebobo]

When I try to downgrade the package I'm given an error stating there is no download candidate/it is outdated or reference by another package.

Not sure what I'm doing wrong here, haha

[jiangcuo]

@jarrodthebobo Can you show your pveversion and vm config? edk2 issue has been resloved by proxmox.

[ruslanvd]

@jiangcuo

Hello. With the new pve-edk2-firmware=3.20230228-4 nothing is solved either :( Maybe this problem is not in the firmware?... My VMs are made as in the wiki, but I have already tried all configuration options. But still, I can't access the VM either from the console or from the web UI. Here are the current configs:

root@node2:~ # pveversion --verbos proxmox-ve: 8.0.1 (running kernel: 5.10.110-rockchip-rk3588) pve-manager: 8.0.3 (running version: 8.0.3/6d727f81c8ca23b5) ceph: 17.2.6-pve1+3 ceph-fuse: 17.2.6-pve1+3 corosync: 3.1.7-pve3 criu: 3.17.1-2 glusterfs-client: 10.3-5 ifupdown: residual config ifupdown2: 3.2.0-1+pmx2 libjs-extjs: 7.0.0-3 libknet1: 1.25-1 libproxmox-acme-perl: 1.4.6 libproxmox-backup-qemu0: 1.4.0 libproxmox-rs-perl: 0.3.0 libpve-access-control: 8.0.3 libpve-apiclient-perl: 3.3.1 libpve-common-perl: 8.0.5 libpve-guest-common-perl: 5.0.3 libpve-http-server-perl: 5.0.3 libpve-rs-perl: 0.8.3 libpve-storage-perl: 8.0.1 libspice-server1: 0.15.1-1 lvm2: 2.03.16-2 lxc-pve: 5.0.2-4 lxcfs: 5.0.3-pve3 novnc-pve: 1.4.0-2 proxmox-backup-client: 2.99.0-1 proxmox-backup-file-restore: 2.99.0-1 proxmox-kernel-helper: 8.0.2 proxmox-mail-forward: 0.2.0 proxmox-mini-journalreader: 1.4.0 proxmox-widget-toolkit: 4.0.5 pve-cluster: 8.0.1 pve-container: 5.0.3 pve-docs: 8.0.3 pve-edk2-firmware: 3.20230228-4 pve-firewall: 5.0.2 pve-firmware: 3.6-4 pve-ha-manager: 4.0.2 pve-i18n: 3.0.4 pve-qemu-kvm: 8.0.2-3 pve-xtermjs: 4.16.0-3 qemu-server: 8.0.6 smartmontools: 7.3-1+b1 spiceterm: 3.3.0 swtpm: 0.8.0+pve1 vncterm: 1.8.0 zfsutils-linux: 2.1.12-pve1

qm showcmd 105 /usr/bin/kvm -id 105 -name 'ubu,debug-threads=on' -no-shutdown -chardev 'socket,id=qmp,path=/var/run/qemu-server/105.qmp,server=on,wait=off' -mon 'chardev=qmp,mode=control' -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' -mon 'chardev=qmp-event,mode=control' -pidfile /var/run/qemu-server/105.pid -daemonize -smbios 'type=1,uuid=fd451ed7-b91d-4faf-8c29-d610ffa14f43' -drive 'if=pflash,unit=0,format=raw,readonly=on,file=/usr/share/pve-edk2-firmware//AAVMF_CODE.ms.fd' -drive 'if=pflash,unit=1,id=drive-efidisk0,format=qcow2,file=/var/lib/vz/images/105/vm-105-disk-0.qcow2' -smp '2,sockets=1,cores=2,maxcpus=2' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' -vnc 'unix:/var/run/qemu-server/105.vnc,password=on' -cpu host -m 2048 -object 'iothread,id=iothread-virtioscsi0' -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pcie.0,addr=0x1e' -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pcie.0,addr=0x1f' -device 'pci-bridge,id=pci.3,chassis_nr=3,bus=pcie.0,addr=0x5' -device 'qemu-xhci,id=qemu-xchi' -device usb-tablet -device usb-kbd -readconfig /usr/share/qemu-server/pve-aarch64.cfg -chardev 'socket,id=serial0,path=/var/run/qemu-server/105.serial0,server=on,wait=off' -serial chardev:serial0 -device 'virtio-gpu,id=vga,bus=pcie.0,addr=0x2' -chardev 'socket,path=/var/run/qemu-server/105.qga,server=on,wait=off,id=qga0' -device 'virtio-serial,id=qga0,bus=pcie.0,addr=0x8' -device 'virtserialport,chardev=qga0,name=org.qemu.guest_agent.0' -device 'virtio-serial,id=spice,bus=pcie.0,addr=0x9' -chardev 'spicevmc,id=vdagent,name=vdagent' -device 'virtserialport,chardev=vdagent,name=com.redhat.spice.0' -spice 'tls-port=61001,addr=127.0.0.1,tls-ciphers=HIGH,seamless-migration=on' -device 'virtio-balloon-pci,id=balloon0,bus=pcie.0,addr=0x3,free-page-reporting=on' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:3b8a83fffae7' -device 'virtio-scsi-pci,id=virtioscsi0,bus=pcie.3,addr=0x1,iothread=iothread-virtioscsi0' -drive 'file=/var/lib/vz/images/105/vm-105-disk-1.qcow2,if=none,id=drive-scsi0,format=qcow2,cache=none,aio=io_uring,detect-zeroes=on' -device 'scsi-hd,bus=virtioscsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,bootindex=101' -device 'virtio-scsi-pci,id=virtioscsi1,bus=pcie.3,addr=0x2' -drive 'file=/var/lib/vz/template/iso/ubuntu-22.04.2-live-server-arm64.iso,if=none,id=drive-scsi1,media=cdrom,aio=io_uring' -device 'scsi-cd,bus=virtioscsi1.0,channel=0,scsi-id=0,lun=1,drive=drive-scsi1,id=scsi1,bootindex=100' -netdev 'type=tap,id=net0,ifname=tap105i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown' -device 'virtio-net-pci,mac=C2:70:90:80:D1:53,netdev=net0,bus=pcie.0,addr=0xb,id=net0,rx_queue_size=1024,tx_queue_size=1024,bootindex=102' -machine 'type=virt+pve0'

[titogrima]

Hi!

I installed pve-edk2-firmware_3.20230228-4_all.deb package download directly from Proxmox 8 repo and not work wget http://download.proxmox.com/debian/pve/dists/bookworm/pve-no-subscription/binary-amd64/pve-edk2-firmware_3.20230228-4_all.deb dpkg -i pve-edk2-firmware_3.20230228-4_all.deb

If downgrade again to pve-edk2-firmware_3.20220526-1_all.deb work without any problem again

@jarrodthebobo try to downgrade to pve-edk2-firmware_3.20220526-1_all.deb package and create de symbolink link

Downgrade edk2 wget http://download.proxmox.com/debian/pve/dists/bullseye/pve-no-subscription/binary-amd64/pve-edk2-firmware_3.20220526-1_all.deb dpkg -i pve-edk2-firmware_3.20220526-1_all.deb

create symboly link cd /usr/share/pve-edk2-firmware ln -s AAVMF_CODE.fd AAVMF_CODE.ms.fd

and execute this comand to set gic-version=host and up de VM again I hope useful...

qm set vmid --args "-machine virt,gic-version=host"

in your case should be qm set 105 --args "-machine virt,gic-version=host"

Regards

[ruslanvd]

@titogrima Thank you so much :). A firmware downgrade helped. Old VMs also start normally.

[titogrima]

good!!

And @jiangcuo thank you very much for this port and your work it's awesome for home labs ;)

Regardas

[jiangcuo]

I have tested the lastest pve-edk2-firmware version 3.20230228-4. It works.

I think this issus cause by my mistake .I missed the code about the gic version when I optimized the patch.

Now，New version of qemu-server(8.0.6-1) has upload,You can update it.

[titogrima]

In Orangepi5 with gic-version=host, pve-edk2-firmware version 3.20230228-4 and qemu-server 8.0.6 dont work and with qemu-server 8.0.6-1 neither work but work with pve-edk2-firmware 3.20220526-1 without problems With Raspberry Pi 4 work with pve-edk2-firmware version 3.20230228-4 and qemu-server 8.0.6 without any problem

Maybe this problem only ocurrs with RK3588 SOC

Regards

[jiangcuo]

@itdivan Can you try downgrade edk2?

wget http://download.proxmox.com/debian/pve/dists/bullseye/pve-no-subscription/binary-amd64/pve-edk2-firmware_3.20220526-1_all.deb
dpkg -i pve-edk2-firmware_3.20220526-1_all.deb

[jiangcuo]

follow https://github.com/jiangcuo/Proxmox-Arm64/issues/28#issuecomment-1614667068

[jiangcuo]

1. updated correct pkg in repo
2. updated this issue on wiki