Why must set "-M virt,its=off" to boot windows arm64 iso?

[gang929]

Environment： HOST: aarch64 cpu Host OS: centos kernel 5.4 Guest OS ISO: windows 10 21390 arm64 ISO, windows 11 22621 arm64 ISO Libvirt 7.0 QEMU 5.1

When booting windows Arm64 CD by "-M virt,accel=kvm,gic_version=3,its=off", it can be boot to win Install page. But, with "-M virt,accel=kvm,gic_version=3,its=on" or "-M virt,accel=kvm,gic_version=3", qemu+kvm will be hanged on logo screen after "Press any key to boot from CD or DVD....", not will boot to win Install page.

Why must set "-M virt,its=off" to boot windows arm64?

Because the "ITS" feature is enabled default by QEMU, but it cannot be set by libvirt xml, so, it cannot boot windows arm64 iso from libvirt xml. The root reason is maybe in HOST linux kernel kvm ITS...

Has any one the same issue? How can I troubleshoot it?

[pmsjt]

This was a result of the fact that QEMU creates ITS tables with a length that exactly matches the peripherals attached. In real world hardware, the ITS size is always aligned to the next power of 2 in length. Windows had a dependency on this practical (admittedly implementation dependent) aspect of the ITS length. This has been addressed and current build of Windows (available through the Windows Insiders program) can boot perfectly under the current QEMU with ITS ON.

Please note that some more recent builds of QEMU have somewhat of a reverse bug: ITS=off will disable the ITS in the emulator but keep it in the ACPI tables, which is wrong. If you are using such a build of QEMU you must use ITS=on (and use a recent build of Windows).

[pmsjt]

Please note that this problem is not in any way related to [kvm-guest-drivers-windows]. The first problem was in Windows and the second in QEMU itself. For that reason alone, I think the project owners should close this issue.

If you want to get heads-up on Arm64 QEMU problems such as this, follow me on Twitter (at)itanium_guy. I use QEMU to work on Arm64 Windows on a daily basis, with configs varying from QCG, KVM, HVF...

[kroese]

@pmsjt I have a container for running Windows for ARM64 via QEMU ( https://github.com/dockur/windows-arm ). On x86 I use all kinds of optimizations for performance (like enabling the Hyper-V Enlightenments, etc). But on ARM64 none of them work. Do you have any tips on qemu/kvm flags that would be useful to improve performance for Windows on ARM64? Or can you show the commandline that you use yourself so I can check if mine is the same?

[pmsjt]

There is some ambiguity on the question you asked. Are you trying to run Arm64 on an x64 machine and somehow expecting it to be fast / hypervisor accelerated? That is not possible. If you are asking about running Arm64 Windows under QEMU on an Arm64 Linux host using KVM, that totally works and the VM will run at near-native speed. Same goes for a Macintosh host (accel HVF). If you are talking about using Win32 QEMU (QEMU running on a Windows host) and using Window's Hypervisor Platform (accel WHPX), then that will work for x86_64 but still not for Arm64. This is for 2 reasons: 1) The hypervisor platform hasn't been available for Arm64 till very recently - it just showed up in the Windows Preview builds; 2) QEMU hasn't been ported to Arm64 Windows yet (and said port will require completing the needed steps to it can consume the brand-new Arm64 hypervisor platform).

[kroese]

I was asking about running Arm64 Windows under QEMU on an Arm64 Linux host using KVM. I am the developer of the container linked above, and it works just fine on a Raspberry Pi5 for example on native speed.

Its just that on the x64 version of that container, I use some rare / undocumented qemu flags for optimal performance. I was just curious if you knew any similar flags for qemu-aarch64 so that I could add them to the project. For example, I have no clue if setting flags like gic_version, secure, its could influence performance. As you seem pretty knowledgeable about the subject, I was just wondering what flags you used yourself.

[pmsjt]

Well, there is only so much a Pi5 can achieve. These will be slower than the slowest and meekest Surface ProX. Keep your expectations at bay. There are not that many silver bullets here. You can try GIC v2 instead of v3. As far as I know the Pi is v2 and emulating v3 (especially with ITS) will be more work for QEMU. Them the normal stuff: Use virtio for as much as you can, instead of using hardware peripherals emulation. The thing that is costing you more performance is likely lack of RAM and, for that, what you can do is to shave as much RAM from the host as you can but, as with every device with limited resources, there are diminishing returns.

[kroese]

The Pi5 was just an example, the container runs on every possible ARM device, like high-end Ampere servers. So I currently set gic_version=host so that it always emulates the same version as the host.

As for the VirtIO devices, yes I use them for the disks (iSCSI), the network, etc.

So it seems there is not much left to optimize. The only VirtIO driver that I wanted I could use is the virtio-vga device. Currently I need to use either ramfb to have display during Windows boot and installation, and virtio-gpu afterwards. It would be really nice if qemu-aarch64 had the virtio-vga device so that it would show display during boot without having to switch to the ramfb device.

In any case, thanks for your help!

[driver1998]

VGA will not work on ARM KVM for some reason unfortunately, there is a similar hybrid device virtio-ramfb in downstream QEMU forks (IIRC the UTM QEMU fork have it) but it is never accepted upstream.

https://www.mail-archive.com/qemu-devel@nongnu.org/msg789592.html