Ubuntu 24.04 Host and Guest on QEMU - Following steps but can't even boot into the iso

[EdgeSync]

Hi,

I am trying to get AMDSEV set up and working on my server, but I can't seem to get it let me build the VM. I think there is some steps or clarifications missing in the docs, hopefully someone will be able to help point me where it is going wrong.

Physical Server: HPE Proliant DL325 Gen 11.    
AMD SEV is enabled in BIOS.      
Host OS: Ubuntu 24.04.       
Kernel: 6.9.0-rc7-snp-host-05b10142ac6a       

I am trying to find my way through the steps on the repo

https://github.com/AMDESE/AMDSEV/tree/tio?tab=readme-ov-file        
https://github.com/AMDESE/AMDSEV/tree/master       

1. Download the repo

2. Switch to snp-latest branch

3. Run ./build.sh

4. run ./install.sh

5. reboot host + boot to snp-host kernel

6. validate all is okay with snphost tool

   ~/snphost/target/debug# ./snphost ok
   [ PASS ] - AMD CPU
   [ PASS ]   - Microcode support
   [ PASS ]   - Secure Memory Encryption (SME)
   [ PASS ]     - SME: Enabled in MSR
   [ PASS ]   - Secure Encrypted Virtualization (SEV)
   [ PASS ]     - Encrypted State (SEV-ES)
   [ PASS ]       - SEV-ES INIT: Enabled
   [ PASS ]     - SEV INIT: SEV is INIT, but not currently running a guest
   [ PASS ]     - Secure Nested Paging (SEV-SNP)
   [ PASS ]       - VM Permission Levels
   [ PASS ]         - Number of VMPLs: 4
   [ PASS ]       - SNP: Enabled in MSR
   [ PASS ]       - SEV Firmware Version: Sev firmware version: 1.55
   [ PASS ]       - SNP INIT: SNP is INIT
   [ PASS ]     - Physical address bit reduction: 6
   [ PASS ]     - C-bit location: 51
   [ PASS ]     - Number of encrypted guests supported simultaneously: 1006
   [ PASS ]     - Minimum ASID value for SEV-enabled, SEV-ES disabled guest: 16
   [ PASS ]     - Reading /dev/sev: /dev/sev readable
   [ PASS ]     - Writing /dev/sev: /dev/sev writable
   [ PASS ]   - Page flush MSR: DISABLED
   [ PASS ] - KVM supported: API version: 12
   [ PASS ]   - SEV enabled in KVM: enabled
   [ PASS ]   - SEV-ES enabled in KVM: enabled
   [ PASS ]   - SEV-SNP enabled in KVM: enabled
   [ PASS ] - Memlock resource limit: Soft: 16778661888 | Hard: 16778661888
   [ PASS ] - RMP table addresses: Addresses: 760217600 - 1306525695
   [ PASS ] - RMP INIT: RMP is INIT
   [ PASS ] - Comparing TCB values: TCB versions match
   
   Platform TCB version:
   TCB Version:
   Microcode:   72
   SNP:         21
   TEE:         0
   Boot Loader: 9
   
   Reported TCB version:
   TCB Version:
   Microcode:   72
   SNP:         21
   TEE:         0
   Boot Loader: 9

7. cd into AMDSEV folder again

8. create a qcow2 disk qemu-img create -f qcow2 ubuntu_2404_vm.qcow2 30G

9. cp the OVMF_VARS.fd file into the current folder: cp AMDSEV/usr/local/share/qemu/OVMF_VARS.fd OVMF_VARS.fd

10. try boot the iso launch-qemu.sh -hda ubuntu_2404_vm.04.qcow2 -cdrom ubuntu-24.04-live-server-amd64.iso

11. Get's stuck after I select the "Try or Install Ubuntu Server"

I have attached the stdout.log - maybe it can point where the issue is - I have absolutely no idea why it won't continue on to the install process. stdout.log

[tlendacky]

I would recommend using https://github.com/AMDESE/AMDSEV/tree/snp-latest

You are likely not seeing any kernel output because the serial console is not configured. When you get to the grub menu, edit the "Try and Install Ubuntu Server" entry and add console=ttyS0,115200n8 to the command line and see if you get some kernel output.

[EdgeSync]

Hey @tlendacky - thanks for getting back.

I've tried your suggestion, still getting stuck on the same part.

I've also tried to get it to boot a premade vm (made an ubuntu 2404 vm with qemu, installed the snp-guest kernel, then point the ./launch-qemu.sh to use this hard disk file e.g. ./launch-qemu.sh -hda ./working_ubuntu_vm.qcow2 -sev-snp) it just boots into the uefi shell, it seems the hard disk isn't recognized by uefi. No filesystem, only a single block device.The disk launches fine with qemu or virsh. I'm not sure if this is relevant, but just odd behavior.

I'm feeling in way over my head on this project hehe

[tlendacky]

There should be a file named stdout.log in the directory where you issued the launch-qemu.sh command. The first line of that file will be the qemu command used. Please post that here for the SNP run so we can look at it.

[EdgeSync]

Okay good news - i've been able to get a little further and boot a VM that is SEV enabled, but i've used virt-install to build and launch the VM as opposed to launch-qemu.sh script. I'm sure I'm very likely doing something wrong with the launch-qemu.sh script, confusing arguments or not using it correctly. Actually I think it wasn't pulling the needed OVMF files from where it was supposed to.

Anways, this is what I ended up going with.

(as root)

virt-install --name U2404 --ram 8192 --disk path=/var/kvm/machines/U2404.qcow2,size=40 --boot loader=/usr/share/OVMF/OVMF_CODE_4M.fd,loader.secure='no',loader.type=pflash,nvram=/var/lib/libvirt/qemu/nvram/U2404.fd --vcpus 2 --os-variant ubuntu24.04 --network bridge=virbr0 --graphics none --console pty,target_type=serial --location /media/isos/u2404.iso,kernel=casper/vmlinuz,initrd=casper/initrd --extra-args 'console=ttyS0,115200n8'

Once ubuntu was installed and running, I installed the snp-guest kernel on the vm and shut it down.

Then modified the virsh XML config file (/etc/libvirt/qemu/U2404.xml) to add in these keys:

<memoryBacking>
  <locked/>
</memoryBacking>

and

<launchSecurity type='sev'>
  <cbitpos>51</cbitpos>
  <reducedPhysBits>1</reducedPhysBits>
  <policy>0x0003</policy>
 </launchSecurity>

Then booted the VM into the snp-guest kernel and now my VM is SEV enabled, per snpguest tool:

 root@ubuntu2404-sev-test:~# ./snpguest/target/debug/snpguest ok
[ PASS ] - SEV: ENABLED
[ FAIL ] - SEV-ES: DISABLED
[ FAIL ] - SNP: DISABLED
[ PASS ] - Optional Features statuses:
[ PASS ]  - VTOM: DISABLED
[ PASS ]  - ReflectVC: DISABLED
...
...

I'll keep digging to get the SEV-ES and SNP set up.

@tlendacky thanks for the responses. I'll compare whats in the command in stdout.log versus what command virsh is actually running, I think that'll point me where the exact problem was when using the launch-qemu.sh script.

I'll mark this issue as closed!

[tlendacky]

> <launchSecurity type='sev'>
>   <cbitpos>51</cbitpos>
>   <reducedPhysBits>1</reducedPhysBits>
>   <policy>0x0003</policy>
>  </launchSecurity>

There isn't any libvirt SNP support, yet, so this will always try to launch an SEV or SEV-ES (setting bit 2 of the policy) guest, which is the reason for the launch-qemu.sh script.

Actually I think it wasn't pulling the needed OVMF files from where it was supposed to.

The build script actually builds and installs Qemu and OVMF in the locations used by the launch-qemu.sh script. So if you followed the README for the snp-latest branch, everything should just work with the launch-qemu.sh script.

[EdgeSync]

@tlendacky Ah okay - that is good information to know. Really appreciate the guidance so far - you are a champ. Thanks bud.

[EdgeSync]

Last update @tlendacky

Okay I've found what I did wrong, and it's such a stupid mistake on my part. I created the hard disk file as a qcow2 file, and named it stupidly (ubuntu_2404_sev.img) because hey, file extensions don't matter...But they do matter in this case - because that makes the launch-qemu.sh script try and load the disk as raw format.
-drive file=/var/kvm/machines/ubuntu_2404_sev.img,if=none,id=disk0,format=raw

Renaming it to "ubuntu_2404_sev.qcow2" - and now the launch script loads it with the format qcow2. -drive file=/var/kvm/machines/U2404.qcow2,if=none,id=disk0,format=qcow2

I wouldn't have caught that without comparing the command in stdout.log as you had suggested.

Happy to say that the VM is now loading, and SEV-SNP is enabled.

Some days my brain doesn't brain so well I guess. Hopefully this issue/comments will be useful for others on what NOT to do.