microsoft / WSL

Issues found on WSL
https://docs.microsoft.com/windows/wsl
MIT License
17.25k stars 812 forks source link

Start CPU at EL2 in ARM bootloader when nestedVirtualization=true #9794

Open sirredbeard opened 1 year ago

sirredbeard commented 1 year ago

Describe the solution you'd like

The EFI bootloader on WSL on ARM starts the CPU at EL1.

[ 0.255558] CPU: All CPU(s) started at EL1

This prevents the WSL kernel from enabling HYP mode:

[hayden@~]$ dmesg | grep -i kvm
[    0.528909] kvm [1]: HYP mode not available

And this prevents utilizing KVM on WSL on ARM, with a custom kernel built with KVM enabled:

[hayden@~]$ qemu-system-aarch64 -machine virt -cdrom ubuntu-22.04.2-live-server-arm64.iso -net nic -net user -boot d -m 1024 -bios /usr/share/qemu/linuxboot_dma.bin -enable-kvm
Could not access KVM kernel module: No such file or directory
qemu-system-aarch64: failed to initialize kvm: No such file or directory
:~ # qemu-system-aarch64 -m 2048 -cpu cortex-a57 -smp 2 -M virt -bios /usr/share/qemu/qemu-uefi-aarch64.bin -drive if=none,file=openSUSE-Tumbleweed-ARM-JeOS-efi.aarch64.raw,id=hd0  -device virtio-blk-device,drive=hd0 -enable-kvm
Could not access KVM kernel module: No such file or directory
qemu-system-aarch64: failed to initialize kvm: No such file or directory

The nestedVirtualization setting in .wslconfig should boot the CPU as EL2 (or, optionally, enabling HYP mode), enabling KVM on WSL on ARM.

Describe alternatives you've considered

Without KVM, virtualization can still be achieved in usermode, but without KVM acceleration.

elsaco commented 1 year ago

@sirredbeard you might want EL2 instead, that's the execution context for nesting on ARM processors.

This is output on a raspberry pi 4 with kvm mode enabled:

CPU: All CPU(s) started at EL2
kvm [1]: Hyp mode initialized successfully

and WSL kernel with kvm enabled:

CPU: All CPU(s) started at EL1
kvm [1]: HYP mode not available

Does EL0 run in hypervisor mode?

sirredbeard commented 1 year ago

I believe you are correct, @elsaco. I went back to the ARM documentation (still learning) and updated the request. Thank you.