search

rivosinc / salus

Risc-V hypervisor for TEE development
92 stars 25 forks source link

Improve boot-time #303

Closed glg-rv closed 1 year ago

glg-rv commented 1 year ago

We are currently walking TWO times the page tables for each PTE we map.

One when we lock, the other when we map. This is the main reason why our paging code is so slow.

This patch addresses both issues for the hypervisor mapping case.

This is just the first step to make our paging code faster. On my machine this patch reduces the boot of salus (real time) from 24 seconds to 7 seconds.

glg-rv commented 1 year ago

That is a great result. How did you measure it?

time NCPU=4 QEMU=../qemu make run_tellus at third run, to avoid bazel getting in the way. Not scientific but the difference is so big doesn't require it.

EDIT: in the CI checks the QEMU test time passed from the usual 1m30 to about 25s (and that's even less scientific)

atishp04 commented 1 year ago

Awesome. Thanks for working on this. We have been carrying an out-of-tree qemu patch to reduce the PCI bar space in virt machine to 2GB to improve the boot time.

I see similar improvement on booting Linux while booting 2 cpus.

Without the PCI hack: $time scripts/run_buildroot.sh real 1m37.615s user 2m48.265s sys 0m0.522s

Without the PCI hack + this series: real 0m36.997s user 0m48.935s sys 0m0.376s

With PCI hack, this series improved the boot time(real time) from ~35sec to ~17sec.