Windows arm64 virtual machine?

[jeremyd2019]

As you may have heard, Windows 11 is requiring the presence of TPM 2.0, and I'm trying to provide it one via qemu on aarch64. I followed the instructions from https://qemu-project.gitlab.io/qemu/specs/tpm.html.

mkdir /tmp/mytpm1
swtpm socket --tpmstate dir=/tmp/mytpm1 \
  --ctrl type=unixio,path=/tmp/mytpm1/swtpm-sock \
  --log level=20

  -chardev socket,id=chrtpm,path=/tmp/mytpm1/swtpm-sock \
  -tpmdev emulator,id=tpm0,chardev=chrtpm \
  -device tpm-tis-device,tpmdev=tpm0 \

I saw #223 and updated to EDK2 that has TPM support, but Windows still shows that the TPM 2.0 device cannot start, The I/O device is configured incorrectly or the configuration parameters to the driver are incorrect.

Am I missing something? I can see the swtpm log stuff during early boot, but nothing after

[stefanberger]

I don't know much about Windows on aarch64. Maybe someone in the QEMU or edk2 community can help.

[stefanberger]

You could post the log your are getting. Maybe something is visible there. You are using a recent QEMU and EDK2 (EDK2 from QEMU?), right?

[jeremyd2019]

I was hoping there was something silly like "you need to run this script to initialize the TPM before Windows will like it" or something. I'll try to record some logs later. I am using qemu-system-arm from debian bullseye, and just tried with the qemu-efi-aarch64 from debian experimental to be sure it was built with TPM2 and secure boot (including MS keys pre-enrolled)

[stefanberger]

You could try to add running swtpm_setup like in the following commands so that the vTPM has platform and EK certificates, though I would not expect this to solve what you are reporting as "The I/O device is configured incorrectly or the configuration parameters to the driver are incorrect."

mkdir /tmp/mytpm1
swtpm_setup --tpmstate /tmp/myvtpm1 --create-ek-cert --create-platform-cert --tpm2 --overwrite
swtpm socket --tpmstate dir=/tmp/mytpm1 \
  --ctrl type=unixio,path=/tmp/mytpm1/swtpm-sock \
  --log level=20 --tpm2

[jeremyd2019]

OK, here's the output from swtpm socket, and a screenshot of the device manager error. This is after using swtpm_setup this time

swtpm_log.txt

[jeremyd2019]

I did notice that qemu in bullseye is at 5.2 not 6.0, so maybe a newer qemu would help, but I somehow doubt it.

[stefanberger]

regarding the log: The log ends with a bunch of PCR_Extends. The PCR_Extends seem to even touch PCRs 10-14, which is a bit unusual. What I am missing also is that EDK2 sets ownership of the platform hierarchy.

I would work my way backwards from the latest QEMU + accompanying EDK2.

[jeremyd2019]

I get the same results with both qemu-system-arm and qemu-efi-aarch64 from debian experimental.

swtpm-experimental.txt

[stefanberger]

Does EDK2 let you into a menu and is there some sort of TPM 2 support visible in one of those menu items? Can you try to run Linux and see whether that shows proper /dev/tpm0 and /dev/tpmrm0. Just for sanity...

[jeremyd2019]

Does EDK2 let you into a menu and is there some sort of TPM 2 support visible in one of those menu items?

Yes. with the qemu-efi-aarch64 from experimental. And it shows different things after swtpm_setup than it does before.

Can you try to run Linux and see whether that shows proper /dev/tpm0 and /dev/tpmrm0. Just for sanity...

I tried to run debian testing. I couldn't figure out how to get it to see the tpm though (maybe that's the problem). I could see it in the ACPI tables in dmesg output, but any tpm module I tried to load didn't seem to do anything.

swtpm-debian.txt

In this log, I put blank lines before I selected to start booting debian from grub (there was some activity there, so either grub or linux was talking to it), and more blank lines before shutting down.

[stefanberger]

We only support the TIS on QEMU for aarch64. So my guess would be that modprobe tpm_tis should activate the TPM driver on aarch64 as well. Otherwise maybe there's another driver with tis in the name that you could try.

[jeremyd2019]

I did that and it didn't do anything, so I'm guessing something's wrong on linux too

[stefanberger]

Please write with which version of QEMU you tried it...

[jeremyd2019]

qemu-system-arm 1:5.2+dfsg-10+b2 with qemu-efi-aarch64 2021.05-1

[jeremyd2019]

root@debian:~# find /lib/modules -name \*tis\*
/lib/modules/5.10.0-7-arm64/kernel/net/netfilter/xt_statistic.ko
/lib/modules/5.10.0-7-arm64/kernel/drivers/char/tpm/tpm_tis_core.ko
/lib/modules/5.10.0-7-arm64/kernel/drivers/char/tpm/tpm_tis_spi.ko
/lib/modules/5.10.0-7-arm64/kernel/drivers/media/pci/mantis
/lib/modules/5.10.0-7-arm64/kernel/drivers/media/pci/mantis/mantis.ko
/lib/modules/5.10.0-7-arm64/kernel/drivers/media/pci/mantis/mantis_core.ko
root@debian:~# modprobe tpm_tis_core 
root@debian:~# modprobe tpm_tis_spi
root@debian:~# dmesg | grep -i tpm
[    0.000000] efi: SMBIOS 3.0=0x7bed0000 TPMFinalLog=0x78110000 MEMATTR=0x7a855018 ACPI 2.0=0x78100018 MOKvar=0x7908a000 TPMEventLog=0x77c96018 MEMRESERVE=0x780f9298 
[    0.000000] ACPI: TPM2 0x000000007810E818 00004C (v04 BOCHS  BXPCTPM2 00000001 BXPC 00000001)
root@debian:~# ls -l /dev/tpm*
ls: cannot access '/dev/tpm*': No such file or directory

[stefanberger]

I don't have an aarch64 host, so it's bit more difficult to work on aarch64. I may try with some Fedora raw image I downloaded (https://getfedora.org/en/workstation/download/)... though I now looked at recent Fedora builds and they don't seem to have the plain tpm_tis either: https://koji.fedoraproject.org/koji/rpminfo?rpmID=26859144

Let me send an email to Eric Auger from RedHat. He did the aarch64 TPM work.

[jeremyd2019]

Thanks. As you can probably tell, I have no idea where to go with this, and barely have any idea about TPM at all. Just that the Windows aarch64 vm Microsoft provided (https://www.microsoft.com/en-us/software-download/windowsinsiderpreviewARM64) did update to Windows 11 preview previously, but now won't update a fresh copy due to not having TPM 2.0. I am currently trying to "reset" the Windows 11 install I got before and make a clean image I can use to test, but I'd like to be able to update also, especially if the next Windows 11 build decides not to install due to this.

[eauger]

Hi Jeremu, Stefan,

I tried again with a fed33 guest and latest qemu. You should get on guest dmesg something like: [root@localhost boot]# dmesg | grep TPM [ 0.000000] ACPI: TPM2 0x000000083C17E718 00004C (v04 BOCHS BXPC 00000001 BXPC 00000001) [ 7.376889] tpm_tis MSFT0101:00: 2.0 TPM (device-id 0x1, rev-id 1)

By the way on my side I also get this: [ 7.379455] tpm tpm0: A TPM error (256) occurred attempting the self test [ 7.380967] tpm tpm0: starting up the TPM manually

./char/tpm/tpm_tis.c should be the driver in use. Compiled with CONFIG_TCG_TIS. This is not the SPI flavor. I am not sure the right driver is enabled in your linux kernel? As for windows setup, I am not familiar with it unfortunately. in qemu 5.2 you should have the TPM_TIS_DEVICE pieces. I also saw you used the enabled EDK2 FW so I don't have any idea atm. Your swtpm cmd line matches what I am using.

Thanks

Eric

[stefanberger]

@eauger

By the way on my side I also get this: [ 7.379455] tpm tpm0: A TPM error (256) occurred attempting the self test [ 7.380967] tpm tpm0: starting up the TPM manually

This is not core to this problem, nevertheless we need to fix this. Most likely you somehow got an older version of libtpms running than you did before or some other corruption occurred.

Are you starting the VM with libvirt? If so, have a look at /var/log/swtpm/libvirt/qemu/<vm name>-swtpm.log. You may need to remove the TPM state file directory /var/lib/libvirt/swtpm/<vm uuid> and then start the VM. If you started it from command line just clear all files from the directory you passed as tpmstate dir.

[jeremyd2019]

./char/tpm/tpm_tis.c should be the driver in use. Compiled with CONFIG_TCG_TIS. This is not the SPI flavor. I am not sure the right driver is enabled in your linux kernel? As for windows setup, I am not familiar with it unfortunately. in qemu 5.2 you should have the TPM_TIS_DEVICE pieces. I also saw you used the enabled EDK2 FW so I don't have any idea atm. Your swtpm cmd line matches what I am using.

I was just using the weekly iso for Debian testing arm64 netinst. I could try with fedora, but it sounds like you're already doing that. Let me know if it would be helpful for me to try that.

Maybe whatever the error is that you got is related, or maybe Windows is asking swtpm things it doesn't know about (like the unexpected PCRs)?

[stefanberger]

Maybe whatever the error is that you got is related, or maybe Windows is asking swtpm things it doesn't know about (like the unexpected PCRs)?

There are no such errors visible in your logs. I think this is failing on a different level.

[stefanberger]

So for the record. I tried to run an aarch64 VM in emulation mode. I ended up downloading both iso and raw image from here. I could not get into a prompt in user single mode on the raw image. The root account is locked. I could not figure out how to log in otherwise. so I tried to use the cdrom.

To start the swtpm I ran this here:

mkdir /tmp/myvtpm1
/usr/share/swtpm/swtpm-create-user-config-files
swtpm_setup --tpmstate /tmp/myvtpm1 --tpm2 --create-ek-cert --create-platform-cert
while :; do swtpm socket --tpmstate dir=/tmp/myvtpm1 --ctrl type=unixio,path=/tmp/myvtpm1/swtpm-sock --log level=20 --tpm2; done

I started qemu with this command line (in emulation mode):

qemu-system-aarch64 --version
QEMU emulator version 6.0.90 (v5.2.0-6537-gbeb1913858)
Copyright (c) 2003-2021 Fabrice Bellard and the QEMU Project developers

qemu-system-aarch64 \
  -machine virt,gic-version=3  -m 4G  \
  -chardev socket,id=chrtpm,path=/tmp/myvtpm1/swtpm-sock \
  -tpmdev emulator,id=tpm0,chardev=chrtpm \
  -device tpm-tis-device,tpmdev=tpm0 \
  -device virtio-blk-pci,drive=drv0 \
  -drive format=raw,file=./Fedora-Workstation-34-1.2.aarch64.raw,if=none,id=drv0 \
  -drive if=pflash,format=raw,file=/usr/share/qemu/edk2-aarch64-code.fd,readonly=on \
  -display sdl \
  -device ramfb -cpu cortex-a72 \
  -device usb-ehci -device usb-kbd \
  -device usb-mouse -usb -cdrom ./Fedora-Workstation-Live-aarch64-34-1.2.iso \
  -no-acpi

The EDK2 menu looks good to me with TPM 2 support that looks like the one on x86_64. I chose to boot from UEFI Misc Device 3 (PCI 0x4), which is the cdrom. The cdrom unfortunately doesn't have rescue mode or so, so it takes ages to boot even when selecting the installation in basic graphics mode which also tries to start GNOME display Manager. Anyway, this isn't leading anywhere good for me on an x86_64 host...

[jeremyd2019]

I found this blip of documentation. Does any of this require non-default options here or in the EDK2 TPM menu?

All device models, lines or series must implement and be in compliance with the International Standard ISO/IEC 11889:2015 or the Trusted Computing Group TPM 2.0 Library and a component which implements the TPM 2.0 must be present and enabled by default.

The following requirements must be met:

• All TPM configurations must comply with local laws and regulations.

• Firmware-based components that implement TPM capabilities must implement version 2.0 of the TPM specification.

• An EK certificate must either be pre-provisioned to the TPM by the hardware vendor or be capable of being retrieved by the device during the first boot experience.

• It must ship with SHA-256 PCR banks and implement PCRs 0 through 23 for SHA-256. Note that it is acceptable to ship TPMs with a single switchable PCR bank that can be utilized for SHA-256 measurements.

• It must support TPM2_HMAC command.

[stefanberger]

None of these should be a problem. We can provide a certificate using swtpm_setup, so that's there, and the SHA-256 PCR banks are also enabled by default. TPM2_HMAC is also supported. Also, we know it works on Windows in an x86_64 VM. Someone with deeper knowledge would have to investigate this on an aarch64 host and have access to Windows for aarch64, which isn't easy to get as it seems.

[jeremyd2019]

Someone with deeper knowledge would have to investigate this on an aarch64 host and have access to Windows for aarch64, which isn't easy to get as it seems.

I can't help with deeper knowledge, but Windows for aarch64 can be gotten from https://www.microsoft.com/en-us/software-download/windowsinsiderpreviewARM64 with a Microsoft account enrolled in the insider program

[stefanberger]

I posted this now on qemu-devel: https://lists.gnu.org/archive/html/qemu-devel/2021-07/msg07376.html

[stefanberger]

@eauger

By the way on my side I also get this: [ 7.379455] tpm tpm0: A TPM error (256) occurred attempting the self test [ 7.380967] tpm tpm0: starting up the TPM manually

Actually ... this error code 0x100 means:

tssreturncode 0x100
TPM_RC_INITIALIZE - TPM not initialized by TPM2_Startup or already initialized

I don't know what firmware you used but likely it doesn't have TPM 2 support. Per this code here it will then send TPM2_STARTUP() to the TPM and send the selftest command again: https://elixir.bootlin.com/linux/latest/source/drivers/char/tpm/tpm2-cmd.c#L731

As long as the resulting /dev/tpm0 is working it should be fine:

tsspcrread -ha 10
count 1 pcrUpdateCounter 92
 digest legnth 32
 43 02 64 dd 39 79 ee b2 4a 8 cf 86 ac 7f d5 7b
 [...]

[stefanberger]

@jeremyd2019 Did you check whether there are any Windows updates that you could install?

[jeremyd2019]

Unfortunately without a TPM the preview image cannot update to the newer Windows 11 preview. That's why I was looking to emulate a TPM in the first place.

[stefanberger]

the newer Windows 11 preview

So there is a 'newer Windows 11 preview?' Maybe that could solve this issue... maybe.

[jeremyd2019]

Unlikely but I'll give it a try. I'll have to use uupdump.net to get the files, because the official update won't run without a working TPM 2.0 device.

[jeremyd2019]

While waiting for that, I downloaded the x86_64 version of Windows 11 and gave that a try in qemu-system-x86_64 with swtpm. As expected, that's been doing a good amount of talking to swtpm during the install, and it shows as "working properly" in device manager. So whatever is going on is definitely ARM64 specific, not due to Windows 11.

[jeremyd2019]

The latest Windows 11 22000.100 build is still saying "The I/O device is configured incorrectly or the configuration parameters to the driver are incorrect." on ARM64.

[eauger]

PPI is not supported by the TPM TIS device while it is on x86. Could that be an issue?

[stefanberger]

PPI is not supported by the TPM TIS device while it is on x86. Could that be an issue?

That's hard to say without logs or knowing what 'it' is looking for. But PPI is primarily used to set certain state of the TPM following a reboot. It's not required to use the TPM. That error code 10 in the screenshot above is unfortunately a bit too generic.

[eauger]

on ARM we are not supporting TPM 1.2. Just to make sure are you using --tpm2 along with swtpm (or is it the default?). In your first edit, I don't see this option?

[jeremyd2019]

Yes, I'm using --tpm2.

If anybody knows how to get better info from Windows, I'd be happy to try. I noticed they have a TpmDiagnostics tool now, I will try some of those options. I tried PlatformType but it gave an error that no supported TPM2 was found.

[stefanberger]

tpm.msc is an executable you could try. Kernel logs would be better. Maybe there's something in the 'Event Viewer' (left click on windows sign and type 'event viewer')?

[jeremyd2019]

Well, I managed to get this log (in two formats), but I don't know that it means anything to anybody but Microsoft...

dumpfile.csv dumpfile.xml.txt

[jeremyd2019]

Seeing error codes 0x800710DF = ERROR_DEVICE_NOT_AVAILABLE 0x80290401 = TPM_E_PCP_DEVICE_NOT_READY 0x8028400F = TBS_E_TPM_NOT_FOUND

[jeremyd2019]

Here's the original etl file before I tried to convert it with tracerpt TPM.zip

[stefanberger]

We would need someone who either knows the requirements of the device and can tell us what we are missing or someone who can make modifications to QEMU to try things out. Enabling just printf's on reads from the TIS interface or writes to it is one thing to try out. Enabling PPI is another thing. It's hard to guess from these error messages what the cause may be.

[naitaku]

Hi, I'm facing the same issue. I think that the memory range for the TPM 2.0 device may be a problem. It is mapped to 0x0C000000 with the current aarch64 machine, but I think it should be 0xFED40000 according to the TPM TIS specification.

I tried to modify the Qemu source code to fix the memory range, but it didn't work... PlatformBus is mapped to 0x0C00000 in hw/arm/virt.c. I changed it to 0xFED40000 and Qemu was not able to boot with the message, "qemu-system-aarch64: -device tpm-tis-device,tpmdev=tpm0: Platform Bus: Can not fit MMIO region of size 5000".

[stefanberger]

I guess neither edk2 nor Linux mind finding the TIS at a non-standard address but Windows doesn't accept it.

[eauger]

Yes the ARM TPM-TIS-SYSBUS device matches the "tcg,tpm-tis-mmio" compat on Linux. As a platform bus it is allowed to be instantiated everywhere in the memory map (see for instance linux Documentation/devicetree/bindings/security/tpm/tpm_tis_mmio.txt). On ARM VIRT the RAM starts at 1GB and its end depends on the memory option, allowed to go up to TB. So the ARM virt memory map cannot accomodate this fixed 0xFED0.0000 location.

[stefanberger]

It feels like Windows may end up forcing the QEMU implementation of a TPM interface on another bus, possibly SPI or IC2, unless Microsoft adapts their driver. Which other TPM interface drivers than the TIS would work on a real aarch64 system? The following ones seem to be using ACPI in some form:

• https://elixir.bootlin.com/linux/latest/source/drivers/char/tpm/tpm_tis_synquacer.c
• https://elixir.bootlin.com/linux/latest/source/drivers/char/tpm/tpm_tis_i2c_cr50.c

[jeremyd2019]

https://www.notebookcheck.net/Windows-11-Insider-Preview-Build-22000-194-blocks-Virtual-Machines-that-lack-TPM-2-0.562358.0.html

I haven't tried it yet, so I don't know if this really breaks any of the workarounds to run without TPM 2.

[stefanberger]

I haven't tried it yet, so I don't know if this really breaks any of the workarounds to run without TPM 2.

I didn't know there were 'workarounds'...

[jeremyd2019]

There has been a workaround to bypass the check for TPM 2 during setup. Some rather confusing press reports last month suggested they'd remove the setup checks from the ISO installers, allowing install on "unsupported" hardware at your own risk, but potentially updates may not be available for unsupported hardware. Of course, I'm sure all of this was in the context of x86_64, and arm64 is generally not considered other than on hardware that ships with Windows pre-installed.

Previously, they were supposed to exclude virtual machines from the hardware requirements, but that never seemed to happen for arm64 VMs (at least under QEMU).

[ms-mahuber]

I've hit this problem as well - before finding this thread, I've tried various things but device manager always shows the TPM2 device warning as referenced above (Win10).

On my setup I got a Linux aarch64 guest with swtpm TPM2 emulation fully working - without showing the selftest error message at guest boot.

Has there been any traction or new insights on this in the meantime?