project-oak / oak

Meaningful control of data in distributed systems.
Apache License 2.0
1.3k stars 112 forks source link

Do a page state change operation before invoking `PVALIDATE` #4972

Closed andrisaar closed 6 months ago

andrisaar commented 6 months ago

This seems to do the trick and we can now PVALIDATE 2 MiB frames.

This PR contains two related changes:

  1. Try to use 2 MiB pages more aggressively when validating memory. Previously we only tried to use 2 MiB pages if the E820 entry perfectly aligned with 2M boundaries; now, I try to use the minimal amount of 4K pages.
  2. Do an explicit Page State Change operation before attempting to validate memory.

We didn't do PSCs before as they seemed to be unnecessary. However, for whatever reason this meant that we were never able to PVALIDATE using 2M pages. However, if we first issue an explicit PSC using a 2M frame, the PVALIDATE succeeds.

This should drastically cut down the number of PVALIDATE operations we do, at the cost of doing some VMGEXITs. An obvious optimization for the future will be to issue 253 PSCs in one request; right now we issue one request per frame.

Whether this indeed makes the VM use hugepages and how much of an effect this will have is still TBD, but I'm pretty sure it won't make anything worse. And stage0 logs say that it's using larger PVALIDATE steps now:

stage0 INFO: SEV-SNP memory validation complete.
stage0 INFO:   Validated using 2 MiB pages: 511
stage0 INFO:   Validated using 4 KiB pages: 256
stage0 INFO:   Valid state not updated: 0
stage0 INFO:   RMP page size mismatch errors (fallback to 4K): 0

Ref b/330345538

pmcgrath17 commented 6 months ago

Well done!