search

AMDESE / AMDSEV

AMD Secure Encrypted Virtualization
297 stars 85 forks source link

SEV-SNP guest stuck while launching #101

Closed kpadwal closed 2 years ago

kpadwal commented 2 years ago

Hi,

I am trying to launch SEV-SNP guest from an Ubuntu 20.04.2 host. I have followed the instructions from https://github.com/AMDESE/AMDSEV/tree/sev-snp-devel, still, guest is stuck while its launch,

Host kernel:

$ uname -a
Linux cl1-2523 5.19.0-rc6-snp-host-a7065246cf78 #1 SMP PREEMPT_DYNAMIC Wed Jul 27 12:21:35 PDT 2022 x86_64 x86_64 x86_64 GNU/Linux

Guest:

$ uname -a
Linux nvidia-test 5.19.0-rc6-snp-guest-a7065246cf78 #1 SMP PREEMPT_DYNAMIC Wed Jul 27 12:13:03 PDT 2022 x86_64 x86_64 x86_64 GNU/Linux

Trying to launch the guest, qcow2 file has sev-snp guest kernel, ./launch-qemu.sh -hda /var/lib/libvirt/images/ubuntu-sev-snp-20.04.2.qcow2 -sev-snp

Select Item: 0x19
[Bds]OsIndication: 0000000000000000
[Bds]=============Begin Load Options Dumping ...=============
  Driver Options:
  SysPrep Options:
  Boot Options:
    Boot0000: UiApp              0x0109
    Boot0001: UEFI QEMU QEMU HARDDISK            0x0001
    Boot0002: UEFI PXEv4 (MAC:525400123456)              0x0001
    Boot0003: EFI Internal Shell                 0x0001
  PlatformRecovery Options:
    PlatformRecovery0000: Default PlatformRecovery               0x0001
[Bds]=============End Load Options Dumping=============
[Bds]BdsWait ...Zzzzzzzzzzzz...
[Bds]Exit the waiting!
[Bds]Stop Hotkey Service!
[Bds]UnregisterKeyNotify: 000C/0000 Success
[Bds]UnregisterKeyNotify: 0017/0000 Success
[Bds]UnregisterKeyNotify: 0000/000D Success
Memory  Previous  Current    Next
 Type    Pages     Pages     Pages
======  ========  ========  ========
  0A    00000080  00000022  00000080
  09    00000010  0000000D  00000010
  00    00000080  00000030  00000080
  05    00000100  0000003F  00000100
  06    00000100  000000B1  00000100
EmuVariablesUpdatedCallback
FSOpen: Open 'NvVars' Success
Saved NV Variables to NvVars file
[Bds]Booting UEFI QEMU QEMU HARDDISK
 BlockSize : 512
 LastBlock : 3BFFFFF
 Valid efi partition table header
 Valid efi partition table header
 Valid primary and Valid backup partition table
 Partition entries read block success
 Number of partition entries: 128
 start check partition entries
 End check partition entries
 Index : 0
 Start LBA : 800
 End LBA : 1007FF
 Partition size: 100000
 Start : 100000 End : 200FFE00
 Index : 1
 Start LBA : 100800
 End LBA : 3BFF7FF
 Partition size: 3AFF000
 Start : 20100000 End : 77FEFFE00
Prepare to Free Pool
 BlockSize : 512
 LastBlock : 3AFEFFF
FSOpen: Open '\EFI\BOOT\BOOTX64.EFI' Success
[Bds] Expand PciRoot(0x0)/Pci(0x3,0x0)/Scsi(0x0,0x0) -> PciRoot(0x0)/Pci(0x3,0x0)/Scsi(0x0,0x0)/HD(1,GPT,ACD9C6CA-1028-41C3-94EB-86B779129F2F,0x800,0x100000)/\EFI\BOOT\BOOTX64.EFI
BdsDxe: loading Boot0001 "UEFI QEMU QEMU HARDDISK " from PciRoot(0x0)/Pci(0x3,0x0)/Scsi(0x0,0x0)
[Security] 3rd party image[0] can be loaded after EndOfDxe: PciRoot(0x0)/Pci(0x3,0x0)/Scsi(0x0,0x0)/HD(1,GPT,ACD9C6CA-1028-41C3-94EB-86B779129F2F,0x800,0x100000)/\EFI\BOOT\BOOTX64.EFI.
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 7D8E3040
Loading driver at 0x0007D4AA000 EntryPoint=0x0007D4CC000
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 7E648018
ProtectUefiImageCommon - 0x7D8E3040
  - 0x000000007D4AA000 - 0x00000000000D4000
BdsDxe: starting Boot0001 "UEFI QEMU QEMU HARDDISK " from PciRoot(0x0)/Pci(0x3,0x0)/Scsi(0x0,0x0)
EmuVariablesUpdatedCallback
FSOpen: Open 'NvVars' Success
Saved NV Variables to NvVars file
InstallProtocolInterface: 605DAB50-E046-4300-ABB6-3DD810DD8B23 7D559220
FSOpen: Open '\EFI\BOOT\fbx64.efi' Success
FSOpen: Open '\EFI\BOOT\fbx64.efi' Success
FSOpen: Open 'EFI' Success
FSOpen: Open 'ubuntu' Success
FSOpen: Open 'BOOTX64.CSV' Success
FSOpen: Open '\EFI\ubuntu\BOOTX64.CSV' Success
EmuVariablesUpdatedCallback
FSOpen: Open 'NvVars' Success
Saved NV Variables to NvVars file
EmuVariablesUpdatedCallback
FSOpen: Open 'NvVars' Success
Saved NV Variables to NvVars file
EmuVariablesUpdatedCallback
FSOpen: Open 'NvVars' Success
Saved NV Variables to NvVars file
FSOpen: Open '\EFI\ubuntu\shimx64.efi' Success
[Security] 3rd party image[0] can be loaded after EndOfDxe: PciRoot(0x0)/Pci(0x3,0x0)/Scsi(0x0,0x0)/HD(1,GPT,ACD9C6CA-1028-41C3-94EB-86B779129F2F,0x800,0x100000)/\EFI\ubuntu\shimx64.efi.
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 7D662540
Loading driver at 0x0007D2EC000 EntryPoint=0x0007D30E000
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 7D663018
ProtectUefiImageCommon - 0x7D662540
  - 0x000000007D2EC000 - 0x00000000000D4000
[Variable]: Rewritten a preexisting variable(0x00000006) with different attributes(0x00000007) - 605DAB50-E046-4300-ABB6-3DD810DD8B23:MokListRT
[Variable]: Rewritten a preexisting variable(0x00000006) with different attributes(0x00000007) - 605DAB50-E046-4300-ABB6-3DD810DD8B23:MokListXRT
[Variable]: Rewritten a preexisting variable(0x00000006) with different attributes(0x00000007) - 605DAB50-E046-4300-ABB6-3DD810DD8B23:SbatLevelRT
InstallProtocolInterface: 605DAB50-E046-4300-ABB6-3DD810DD8B23 7D39B220
FSOpen: Open '\EFI\ubuntu\grubx64.efi' Success
PixelBlueGreenRedReserved8BitPerColor
ConvertPages: range 1000000 - 3FA2FFF covers multiple entries
ConvertPages: range 1000000 - 3FA2FFF covers multiple entries
CpuDxe: 5-Level Paging = 0
MpInitChangeApLoopCallback() done!
SetUefiImageMemoryAttributes - 0x000000007F2E8000 - 0x0000000000007000 (0x0000000000000008)
SetUefiImageMemoryAttributes - 0x000000007F2E2000 - 0x0000000000006000 (0x0000000000000008)
SetUefiImageMemoryAttributes - 0x000000007F2DB000 - 0x0000000000007000 (0x0000000000000008)
SetUefiImageMemoryAttributes - 0x000000007F2D5000 - 0x0000000000006000 (0x0000000000000008)
SetUefiImageMemoryAttributes - 0x000000007F2C5000 - 0x0000000000010000 (0x0000000000000008)
SetUefiImageMemoryAttributes - 0x000000007F2C0000 - 0x0000000000005000 (0x0000000000000008)
SetUefiImageMemoryAttributes - 0x000000007F2B9000 - 0x0000000000007000 (0x0000000000000008)
SetUefiImageMemoryAttributes - 0x000000007F2B5000 - 0x0000000000004000 (0x0000000000000008)
SetUefiImageMemoryAttributes - 0x000000007F2B0000 - 0x0000000000005000 (0x0000000000000008)

Any ideas on fixing this issue would be greatly appreciated!

Thanks, Kiran

kpadwal commented 2 years ago

Attaching the guest launch log. guest-launch.txt

kpadwal commented 2 years ago

The issue is resolved. I have booted the VM with kernel and initrd.

ShannonSD commented 2 years ago

Hi kpadwal,

I'm facing the same issue as you were. Could you elaborate on how you solved it? Also could you specify which branch you were using as I see the launch-qemu script is different in each.

kpadwal commented 2 years ago

I copied the kernel and initrd of the guest kernel to the hypervisor and then booted the VM with the -kernel, -initrd, and -append arguments.

snp-release-2022-07-27/usr/local/bin/qemu-system-x86_64 -enable-kvm -cpu EPYC-v4

-machine q35 -smp 4,maxcpus=64 -m 2048M,slots=5,maxmem=30G -no-reboot -drive if=pflash,format=raw,unit=0,file=<>/AMDSEV/snp-release-2022-07-27/usr/local/share/qemu/OVMF_CODE.fd,readonly -netdev user,id=vmnic -device virtio-net-pci,disable-legacy=on,iommu_platform=true,netdev=vmnic,romfile= -drive file=/var/lib/libvirt/images/ubuntu-20.04.2_test.qcow2,if=none,id=disk0,format=qcow2 -device virtio-scsi-pci,id=scsi0,disable-legacy=on,iommu_platform=true -device scsi-hd,drive=disk0 -machine memory-encryption=sev0,vmport=off -object sev-snp-guest,id=sev0,cbitpos=51,reduced-phys-bits=1 -kernel <>/guest_kernel/vmlinuz-5.19.0-rc6-snp-guest-a7065246cf78 -append "console=ttyS0 earlyprintk=serial root=/dev/sda2" -initrd <>/guest_kernel/initrd.img-5.19.0-rc6-snp-guest-a7065246cf78 -nographic -monitor pty -monitor unix:monitor,server,nowait