Is there any movement on getting the CUSE TPM interface accepted upstream?

[trevor-vaughan]

I'm hoping to use the swtpm for testing but I haven't seen much traction.

Patching QEMU is certainly possible but it would be nice to be able to vote for this to be accepted upstream as well.

[stefanberger]

I would certainly like to see it accepted as well. Though criticism was around the CUSE interface. Even if the CUSE interface is 'awful', it's implementation of the control channel is done and the control channel is working, so it's not that bad. Besides that, once one goes beyond looking at the bits and pieces of the interface, one doesn't really realize that it's an ioctl interface. I also have support for libvirt for running QEMU VMs with attached CUSE TPM, where sVirt is used to label the CUSE device.

Anyway, any help on the QEMU mailing list would be appreciated., though I am not following this mailing list at the moment.

[trevor-vaughan]

Thanks for the feedback. Could you point me at the latest thread tail by chance? I found several references but I want to make sure I hit the right one.

Frankly, if they want to include it with a big warning stating that the underlying implementation has issues X, Y, and Z would still make it more useful than not having it.

And :+1 for the sVirt labels, that should mitigate a LOT of issues.

[stefanberger]

The tail end of the discussion is approximately this message:

http://lists.nongnu.org/archive/html/qemu-devel/2016-02/msg00143.html

[stefanberger]

If there are issues X, Y, and Z, I would like to know about them...

[trevor-vaughan]

Sure thing. I've looked through the various arguments and they seem to stem more around theoretical issues rather than tested defects so hopefully this won't be a big deal.

[pohly]

More recent discussion here: https://lists.nongnu.org/archive/html/qemu-devel/2016-06/msg04644.html

[stefanberger]

The discussion is about the choice of interface for how QEMU could talk to an external TPM emulator. CUSE is available at least on Ubuntu, Fedora, and RHEL. UnixIO is available on all Linux distributions. The new VTPM proxy driver is likely not available on any distro right now. I have been using the CUSE TPM integrated with QEMU for a while now.

So, if it comes to a choice of interfaces,, I would go for the one that's available on all distros following that some in the QEMU community don't seem to like CUSE: UnixIO (local) sockets. The disadvantage of this communication interface could be that debugging tools for the TPM may not readily work with this interface. So, CUSE has an advantage here if it ever came to that. But CUSE for sure needs root rights on the TPM emulator (which it can then drop anyways).

PS: 'swtpm chardev --vtpm-proxy --tpmstate dir=/tmp' spawns a Linux vTPM proxy device now and listens for commands.

[stefanberger]

@trevor-vaughan The tpm_emulator driver has now been upstreamed to QEMU. It is using swtpm though the UnixIO socket interface.

Can we close this issue?

[trevor-vaughan]

Seems reasonable to me

[stenioaraujo]

@stefanberger Hi, first of all, thank you for the attention you have been giving for the questions.

You said in your last message that the tpm_emulator is upstream to QEMU. I was able to build the swtpm, but I was not able to launch a VM using virt-install for a vTPM. Is there any documentation about tpm_emulator on QEMU? Thank you.

[stefanberger]

There's not support for libvirt for tpm_emulator so far and it may depend on that. Here's a link to some documentation on the QEMU level: https://git.qemu.org/?p=qemu.git;a=blob_plain;f=docs/specs/tpm.txt;hb=6cf38cbf2961c57bd85022cf9adf7ef85dae0f02

[stenioaraujo]

Ohh, That makes sense. I am going to try to use it with QEMU only then. Thank you again.

[stenioaraujo]

@stefanberger

I followed the information in the documentation to use swtpm with QEMU using UnixIO. I used the qemu:command tag in the libvirt XML file to pass the arguments directly to the QEMU command. Unfortunately, the VM won't start. It raises a Failed to connect socket: Permission denied error when using the socket file created. Is it related to the user the qemu process is running as?

mkdir /tmp/myvtpm0
swtpm socket --tpmstate dir=/tmp/myvtpm0 --ctrl type=unixio,path=/tmp/myvtpm0/swtpm-sock --log file=/var/log/swtpm_myvtpm0.log,level=20 -d

  <qemu:commandline>
    <qemu:arg value='-chardev' />
    <qemu:arg value='socket,id=chrtpm,path=/tmp/myvtpm0/swtpm-sock' />
    <qemu:arg value='-tpmdev' />
    <qemu:arg value='emulator,id=tpm0,chardev=chrtpm' />
  </qemu:commandline>

I am sorry if this is not the best place for those questions, point me to a better place if this is not the case. Thank you.

[stefanberger]

libvirt will start QEMU as the qemu user. So you would have to run the following command after starting the swtpm:

sudo chown qemu /tmp/myvtpm0/swtpm-sock

With integration into libvirt, libvirt would setup the socket ownership as needed or use file descriptor passing. For now we have to setup the permissions and ownership using commands ourselves or we extend the unixio type of socket parameter with ownership options (uid=qemu,gid=...,mode=...) so swtpm does it.

[stenioaraujo]

By default my qemu is running as the libvirt+ user. I cant change the ownership of the folder to that.

I edited my /etc/libvirt/qemu.conf so the user and group are root (Just for testing). It still gives me the Permission Denied error. Both socket file and qemu are root`s.

I don`t understand the last part.

Thank you

[stefanberger]

Please also add -device tpm-tis,tpmdev=tpm0 to your parameters:

 <qemu:commandline>
    <qemu:arg value='-chardev'/>
    <qemu:arg value='socket,id=chrtpm,path=/tmp/myvtpm0/swtpm-sock' />
    <qemu:arg value='-tpmdev'/>
    <qemu:arg value='emulator,id=tpm0,chardev=chrtpm'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='tpm-tis,tpmdev=tpm0'/>
  </qemu:commandline>

[stefanberger]

I apologize for accidentally closing this issue.

[stefanberger]

I added support for setting file mode flags (mode=0...) and ownership of the UnixIO socket (uid=...,gid=....) to the parameters that understand type=unixio.

[stenioaraujo]

Sorry for not answering previously.

I am trying to setup a new Environment so I can test it better. The modifications you suggested to my XML didn't work. It still gives me Permission Denied.

[stenioaraujo]

Yeah, Unfortunately the new environment gives me the same error. Even using the UID and GID flags

error: Failed to start domain testing
error: internal error: early end of file from monitor, possible problem: Domain id=15 is tainted: custom-argv
2018-04-03T14:42:54.316025Z qemu-system-x86_64: -chardev socket,id=chrtpm,path=/tmp/myvtpm0/swtpm-sock: Failed to connect socket: Permission denied

I am going to debug the system calls to see if I am to find where exactly the problem is raised.

[stefanberger]

If SELinux is enabled, you may want to try to disable it.

[stenioaraujo]

I am using Ubuntu 16.04.4 LTS, with libvirt 1.3.1 and qemu 2.5.0. I don't have SELinux on my machine. Instead, I tried disabling AppArmor, but it still didn't work.

I ran the virsh start command with strace to identify the error, but I am not able to find anything unusual in it: https://gist.github.com/stenioaraujo/252d14ecc4724fe9dcdd242a5d69f54c (Line 895 shows when the error appears).

My VM's XML file is: https://gist.github.com/stenioaraujo/3ab65bacbe9bcdc4a647f5ca49a7c366

I am wondering if this is really a file permission issue. Even when I am running qemu as root it still happens. It shouldn't right?

[stefanberger]

No, it shouldn't. Can you start it from the command line?

[stenioaraujo]

Using

qemu-system-x86_64 -enable-kvm -name testing -S -machine pc-i440fx-xenial,accel=kvm,usb=off -cpu Haswell-noTSX -drive file=/usr/share/OVMF/OVMF_CODE.fd,if=pflash,format=raw,unit=0,readonly=on -drive file=/var/lib/libvirt/qemu/nvram/testing_VARS.fd,if=pflash,format=raw,unit=1 -m 1024 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid c70c295b-dbf2-40de-8fdb-b520ee3ab560 -no-user-config -nodefaults -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x6.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x6 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x6.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x6.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive file=/local/testing/testing.qcow2,format=qcow2,if=none,id=drive-ide0-0-0 -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive if=none,id=drive-ide0-0-1,readonly=on -device ide-cd,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1  -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 -spice port=5900,addr=127.0.0.1,disable-ticketing,image-compression=off,seamless-migration=on -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0 -chardev spicevmc,id=charredir1,name=usbredir -device usb-redir,chardev=charredir1,id=redir1 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -chardev socket,id=chrtpm,path=/tmp/myvtpm0/swtpm-sock -tpmdev emulator,id=tpm0,chardev=chrtpm -device tpm-tis,tpmdev=tpm0 -msg timestamp=on

I get the following:

2018-04-03T18:10:04.623244Z qemu-system-x86_64: -tpmdev emulator,id=tpm0,chardev=chrtpm: Parameter 'type' expects a TPM backend type
Supported TPM types (choose only one):
 passthrough   Passthrough TPM backend driver

it doesn't look like I had any problem with the swtpm socket.

[stefanberger]

You need to compile QEMU from the git repository. Yours is missing the emulator backend.

[stenioaraujo]

@stefanberger, sorry for taking so long to give a feedback.

I was able to run QEMU directly with SWTPM. The VM recognizes the vTPM, I can create a new EK, take ownership and activate trusted boot (with ima).

I followed the steps to compile the libtpms and swtpm from here.

I compiled the QEMU as suggested.

Then I run my guests as:

mkdir -p ${VTPM_DIR_PATH}; swtpm socket --tpmstate dir=${VTPM_DIR_PATH} --ctrl type=unixio,path=${VTPM_DIR_PATH}/swtpm-sock --log file=${VTPM_DIR_PATH}/vtpm.log,level=20 -d;

qemu-system-x86_64 -hda ${VM_NAME}.qcow2 -boot d -cdrom ${OS_IMAGE} -m ${VM_RAM} -enable-kvm -chardev socket,id=chrtpm,path=${VTPM_DIR_PATH}/swtpm-sock -tpmdev emulator,id=tpm0,chardev=chrtpm -device tpm-tis,tpmdev=tpm0 -device e1000,netdev=net0 -netdev user,id=net0,hostfwd=tcp::${LOCALHOST_SSH_PORT}-:22 -drive file=/usr/share/OVMF/OVMF_CODE.fd,if=pflash,format=raw,unit=0,readonly=on -drive file=$(pwd)/${VM_NAME}_nvram.fd,if=pflash,format=raw,unit=1 -vga qxl -spice port=${SPICE_PORT},disable-ticketing

I setup the TPM and IMA. But my PCR values from 1 to 7 are always 0's Is that a common behavior?

Also, thank you all the patience, I think you might close this issue.

[stefanberger]

@stenioaraujo It may be due to the version of SeaBIOS you are using that the PCRs 0-7 are zero. You may want to checkout SeaBIOS from git and compile it and cp out/bios.bin /usr/shared/seabios/bios-256k.bin (works on Fedora) and add -L /usr/share/seabios -bios bios-256k.bin to the QEMU parameters. Though the latest QEMU git checkout should also have a version that works with the TIS interface (not the CRB at the moment since CRB patches were not part of the build).

[stenioaraujo]

I am using UEFI with OVMF. Would that be the same approach?

[stefanberger]

@stenioaraujo OVMF has only gotten TPM support recently (for VMs) and only for TPM 2. You would have to build it from the sources...

[stenioaraujo]

@stefanberger You mean by following their guide? I had a hard time trying to understand their instruction. I will keep studying it.

Thank you for the help.

[stenioaraujo]

@stefanberger I compiled OVMF with TPM2 Enabled, I compiles libtpms and swtpm from the tpm2 branches.

I started a VM using the OVMF, inside the VM. I ran dmesg | grep -i tpm and I have the following result:

[    0.000000] ACPI: TPM2 0x000000003E752000 00004C (v04 BOCHS  BXPCTPM2 00000001 BXPC 00000001)
[    0.647122] tpm_tis 00:06: 2.0 TPM (device-id 0x1, rev-id 1)

But when I try to run tpm2_listpcrs for example, I get the error message Resource Mgr, resMgr, failed initialization: 0x1. Exiting...

The same happens with a VM with BIOS.

Would that be a swtpm related setting?

I set swtpm as follow:

mkdir -p ${VTPM_DIR_PATH}
swtpm_setup --tpmstate ${VTPM_DIR_PATH} --createek --tpm2
swtpm socket --tpmstate dir=${VTPM_DIR_PATH} --ctrl type=unixio,path=${VTPM_DIR_PATH}/swtpm-sock --log file=${VTPM_DIR_PATH}/vtpm.log,level=20 --tpm2 -d

[stefanberger]

The tpm2_listpcrs related problem may be a question to ask on Intel's TPM2 stack mailing list. I do not think that this is related to the firmware. Though to make sure, are there any error messages printed out by swtpm very early on?

I am running for example Fedora 28 (preview) and have to have the tpm2-abrmd running.

systemctl status tpm2-abrmd

Though I think older versions of Intel's TPM 2 stack needed a different service. I seem to remember starting resourceMgr for an older version of this stack. Did you start it and it died on you?

[stenioaraujo]

'swtpm' starts normally. I didn't have tpm2-abrmd on the VM, I only installed tpm2-tools. So I installed them from source (tpm2-tss, tpm2-abrmd and tpm2-tools). I was able to set tpm2-abrmd up so the service could start. Now I get another error when running tpm2_pcrlist:

** (process:15922): WARNING **: Failed to create connection with service: GDBus.Error:org.freedesktop.DBus.Error.AccessDenied: Rejected send message, 1 matched rules; type="method_call", sender=":1.18" (uid=0 pid=15922 comm="tpm2_pcrlist ") interface="com.intel.tss2.TctiTabrmd" member="CreateConnection" error name="(unset)" requested_reply="0" destination=":1.19" (uid=999 pid=15925 comm="/usr/local/sbin/tpm2-abrmd ")
ERROR: tcti init allocation routine failed for library: "tabrmd" options: "(null)"
ERROR: Could not load tcti, got: "tabrmd"

Probably my installation of tpm2-abrmd wasn't so successful.

The tpm2 mailing list you suggested is tpm2@lists.01.org?

[stefanberger]

Try the following:

systemctl daemon-reload && systemctl start tpm2-abrmd; systemctl status tpm2-abrmd

Yes, I think this is the mailing list for Intel's TPM 2 stack: https://lists.01.org/mailman/listinfo/tpm2

[stenioaraujo]

● tpm2-abrmd.service - TPM2 Access Broker and Resource Management Daemon
   Loaded: loaded (/usr/local/lib/systemd/system/tpm2-abrmd.service; enabled; vendor preset: enabled)
   Active: active (running) since Qua 2018-04-18 17:27:57 BRT; 9min ago
 Main PID: 2301 (tpm2-abrmd)
   CGroup: /system.slice/tpm2-abrmd.service
           └─2301 /usr/local/sbin/tpm2-abrmd

Abr 18 17:27:57 ubuntu systemd[1]: Starting TPM2 Access Broker and Resource Management Daemon...
Abr 18 17:27:57 ubuntu systemd[1]: Started TPM2 Access Broker and Resource Management Daemon.
Abr 18 17:34:58 ubuntu systemd[1]: Started TPM2 Access Broker and Resource Management Daemon.

It still the same result when I use tpm2_pcrlist. I am going to ask there, thank you @stefanberger

[stenioaraujo]

@stefanberger just updating, so you may close the thread:

The setup swtpm + libtpms + QEMU + OVMF + tpm2-tools worked. For that I:

1. Compiled libtpms and swtpm following the instructions in here with: a. ./configure --with-openssl --with-tpm2 when compiling libtpms b. ./configure --with-openssl when compiling swtpm
2. Compiled QEMU with spice support from master to have the support to tpm emulator backend:

   apt install -y libspice-server-dev libspice-protocol-dev libpixman-1-dev
   git clone --recursive https://github.com/qemu/qemu
   pushd qemu
   ./configure --target-list=x86_64-softmmu --enable-spice # For x86_64 platform with -vga qxl spice enabled
   make
   make install
   popd

3. Compiled EDK2. Following the instruction in here.
4. Compiled OVMF following the instructions in here, and setting the flags when building the OVMF as follow: build -D TPM2_ENABLE=TRUE -D SECURE_BOOT_ENABLE=TRUE -D HTTP_BOOT_ENABLE=TRUE
5. Created and started the swtpm socket for tpm2 on the host machine.

   swtpm_setup --tpmstate ${VTPM_DIR_PATH} --createek --tpm2
   swtpm socket --tpmstate dir=${VTPM_DIR_PATH} --ctrl type=unixio,path=${VTPM_DIR_PATH}/swtpm-sock --log file=${VTPM_DIR_PATH}/vtpm.log,level=20 --tpm2 -d

6. Created a new Virtual Machine with QEMU and use OVMF as the UEFI implementation

   
   # Reuse OVMF_CODE but create a copy of the NVRAM file for this VM
   BASE_OVMF_CODE_PATH=${HOME}/edk2/Build/OvmfX64/DEBUG_GCC5/FV/OVMF_CODE.fd # Path to the OVMF_CODE.fd with TPM2_ENABLE enabled
   BASE_NVRAM_PATH=${HOME}/edk2/Build/OvmfX64/DEBUG_GCC5/FV/OVMF_VARS.fd
   cp ${BASE_NVRAM_PATH} ${VM_NAME}_nvram.fd

Create the virtual machine disk

qemu-img create -f qcow2 ${VM_NAME}.qcow2 ${VM_DISK_SIZE}

Create a UEFI flavored virtual machine using QEMU pointing to the socket. This will run in background.

qemu-system-x86_64 -hda ${VM_NAME}.qcow2 -boot d -cdrom ${OS_IMAGE} -m ${VM_RAM} -enable-kvm -chardev socket,id=chrtpm,path=${VTPM_DIR_PATH}/swtpm-sock -tpmdev emulator,id=tpm0,chardev=chrtpm -device tpm-tis,tpmdev=tpm0 -device e1000,netdev=net0 -netdev user,id=net0,hostfwd=tcp::${LOCALHOST_SSH_PORT}-:22 -vga qxl -spice port=${SPICE_PORT},disable-ticketing -drive file=${BASE_OVMF_CODE_PATH},if=pflash,format=raw,unit=0,readonly=on -drive file=$(pwd)/${VM_NAME}_nvram.fd,if=pflash,format=raw,unit=1 &

7. Compiled tpm2-tools within the guest machine, the steps to compile may be seen [here](https://git.lsd.ufcg.edu.br/snippets/58)

Thanks for all the help @stefanberger 

[stefanberger]

@stenioaraujo Thanks for trying this out. A different thread would have been a better place to post it.... I am closing this thread now.