Open TheNetAdmin opened 1 year ago
tlendacky
commented
1 year ago The unsupported event was reported by the guest to the hypervisor. A 0x404 error code indicates that the page was not validated. Are you using the AMDSEV sev-snp-devel branch script to build OVMF or are you building it on your own? If on your own, ensure that you are not building with SMM or Secure Boot enabled.
TheNetAdmin
commented
1 year ago Hi I used the sev-snp-devel branch. I also tried another pre-built edk2 ovmf from https://github.com/retrage/edk2-nightly which was working on other SEV-SNP platforms but gives the same 0x404 error on my platform.
TheNetAdmin
commented
1 year ago And I notice the following command on host doesn't give any output
$ sudo dmesg | grep -i rmp
(empty output)Maybe due to some reason the RMP table was not set up on the host?
And another thing I noticed is that although the mother board (Gigabyte MZ72 HB0) supports 8 DRAM DIMMs, I only inserted four (should be one per channel according to the motherboard manual).
Isthis another possible reason that RMP is not set up?
tlendacky
commented
1 year ago $ dmesg | grep -i sev [ 0.656071] SEV-SNP: RMP table physical address 0x0000000054000000 - 0x00000000747fffff [ 2.536187] ccp 0000:42:00.1: sev enabled [ 2.598793] ccp 0000:42:00.1: SEV firmware update successful [ 4.378148] ccp 0000:42:00.1: SEV API:1.51 build:3 [ 4.378157] ccp 0000:42:00.1: SEV-SNP API:1.51 build:3 [ 4.409337] SEV supported: 410 ASIDs [ 4.409337] SEV-ES and SEV-SNP supported: 99 ASIDs
Hmmm... this is from your earlier comment. Are you sure that your dmesg just hasn't been cleared? Otherwise, you wouldn't even be able to start launching an SNP guest.
The number of DIMMs installed does not affect whether the RMP will be allocated or not.
If you're comfortable with modifying OVMF code, you can try adding some debug statements to QemuFlashInitialize() and QemuFlashDetected() in OvmfPkg/QemuFlashFvbServicesRuntimeDxe/QemuFlash.c. Just use a simple: DEBUG ((DEBUG_INFO, "*** DEBUG: %a:%u\n", func, LINE)); to track progress and see where the error occurs (assuming it reaches QemuFlashInitialize()).
TheNetAdmin
commented
1 year ago $ dmesg | grep -i sev [ 0.656071] SEV-SNP: RMP table physical address 0x0000000054000000 - 0x00000000747fffff [ 2.536187] ccp 0000:42:00.1: sev enabled [ 2.598793] ccp 0000:42:00.1: SEV firmware update successful [ 4.378148] ccp 0000:42:00.1: SEV API:1.51 build:3 [ 4.378157] ccp 0000:42:00.1: SEV-SNP API:1.51 build:3 [ 4.409337] SEV supported: 410 ASIDs [ 4.409337] SEV-ES and SEV-SNP supported: 99 ASIDs
Hmmm... this is from your earlier comment. Are you sure that your dmesg just hasn't been cleared? Otherwise, you wouldn't even be able to start launching an SNP guest.
Oh I figured that I disabled the IOMMU in BIOS and the RMP table message was not present. Now I enabled the IOMMU, the RMP message is back, but the SEV-SNP guest still reports the same invalid argument error
The number of DIMMs installed does not affect whether the RMP will be allocated or not.
If you're comfortable with modifying OVMF code, you can try adding some debug statements to QemuFlashInitialize() and QemuFlashDetected() in OvmfPkg/QemuFlashFvbServicesRuntimeDxe/QemuFlash.c. Just use a simple: DEBUG ((DEBUG_INFO, "* DEBUG: %a:%u\n", func, LINE**)); to track progress and see where the error occurs (assuming it reaches QemuFlashInitialize()).
Thank you for this detailed guide! I'm working on this debugging and will report what I find.
tlendacky
commented
1 year ago DEBUG ((DEBUG_INFO, "* DEBUG: %a:%u\n", func, LINE**));
Github changed what I typed, it should be:
DEBUG ((DEBUG_INFO, "*** DEBUG: %a:%u\n", __func__, __LINE__));
aep
commented
1 year ago ? If on your own, ensure that you are not building with SMM or Secure Boot enabled.
is there a deeper reason this is incompatible? a customer requires secureboot
tlendacky
commented
1 year ago The SMM support of Qemu/KVM requires the hypervisor to be able to change vCPU register state, which is not possible under SEV-ES/SEV-SNP.
aep
commented
1 year ago sorry, i meant to ask for secureboot only. My colleague found that disabling this made it work:
diff --git a/OvmfPkg/PlatformPei/Platform.c b/OvmfPkg/PlatformPei/Platform.c
index 148240342b..c292a44def 100644
--- a/OvmfPkg/PlatformPei/Platform.c
+++ b/OvmfPkg/PlatformPei/Platform.c
@@ -222,9 +222,9 @@ ReserveEmuVariableNvStore (
VariableStore = (EFI_PHYSICAL_ADDRESS)(UINTN)PlatformReserveEmuVariableNvStore ();
PcdStatus = PcdSet64S (PcdEmuVariableNvStoreReserved, VariableStore);
- #ifdef SECURE_BOOT_FEATURE_ENABLED
- PlatformInitEmuVariableNvStore ((VOID *)(UINTN)VariableStore);
- #endif
+// #ifdef SECURE_BOOT_FEATURE_ENABLED
+// PlatformInitEmuVariableNvStore ((VOID *)(UINTN)VariableStore);
+// #endif
ASSERT_RETURN_ERROR (PcdStatus);
}DEBUG ((DEBUG_INFO, "* DEBUG: %a:%u\n", func, LINE**));
Github changed what I typed, it should be:
DEBUG ((DEBUG_INFO, "*** DEBUG: %a:%u\n", __func__, __LINE__));
I inserted a few such debug prints to the file, and located the lines that caused the invalid argument error:
These two lines failed during the first iteration of this loop.
And I also noticed my current CPU (7443P) microcode is as follows
$ sudo dmesg | grep code
...
[ 1.080107] microcode: CPU0: patch_level=0x0a00115d
...This patch_level does not match anyone described in:
https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/amd-ucode/README
Is the micro code a concern in this invalid argument error?
Is there any idea that I can further investigate this issue?
tlendacky
commented
1 year ago sorry, i meant to ask for secureboot only. My colleague found that disabling this made it work:
diff --git a/OvmfPkg/PlatformPei/Platform.c b/OvmfPkg/PlatformPei/Platform.c index 148240342b..c292a44def 100644 --- a/OvmfPkg/PlatformPei/Platform.c +++ b/OvmfPkg/PlatformPei/Platform.c @@ -222,9 +222,9 @@ ReserveEmuVariableNvStore ( VariableStore = (EFI_PHYSICAL_ADDRESS)(UINTN)PlatformReserveEmuVariableNvStore (); PcdStatus = PcdSet64S (PcdEmuVariableNvStoreReserved, VariableStore); - #ifdef SECURE_BOOT_FEATURE_ENABLED - PlatformInitEmuVariableNvStore ((VOID *)(UINTN)VariableStore); - #endif +// #ifdef SECURE_BOOT_FEATURE_ENABLED +// PlatformInitEmuVariableNvStore ((VOID *)(UINTN)VariableStore); +// #endif ASSERT_RETURN_ERROR (PcdStatus); }
Just FYI, this has been reported and is being worked on upstream in the edk2-devel mailing list.
tlendacky
commented
1 year ago DEBUG ((DEBUG_INFO, "* DEBUG: %a:%u\n", func, LINE**));
Github changed what I typed, it should be:
DEBUG ((DEBUG_INFO, "*** DEBUG: %a:%u\n", __func__, __LINE__));I inserted a few such debug prints to the file, and located the lines that caused the
invalid argumenterror:These two lines failed during the first iteration of this loop.
I haven't seen this issue before, so I'm not sure why that is happening.
And I also noticed my current CPU (7443P) microcode is as follows
$ sudo dmesg | grep code ... [ 1.080107] microcode: CPU0: patch_level=0x0a00115d ...This patch_level does not match anyone described in:
https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/amd-ucode/README
Is the micro code a concern in this
invalid argumenterror?
Shouldn't be, but you can always download that firmware file and place it in /lib/firmware/amd-ucode/ and see if that helps.
Is there any idea that I can further investigate this issue?
I would say to wait for the next level of SNP code that will be submitted upstream and try that. There should be corresponding updates to the stable-commits file.
When I try to launch an SEV-SNP VM I got the following error during the guest boot, and I'm wondering what might went wrong in my setup?
And while QEMU reports this error, the host Linux reports
I used the latest sev-snp commit a480a514798b9d226b5aca9f96ce47c9458f504c
My hardware is
And the host Linux system is set up as
And the full log of the QEMU output