search

stefanberger / swtpm

Libtpms-based TPM emulator with socket, character device, and Linux CUSE interface.
Other
577 stars 143 forks source link

Error starting domain: internal error: Could not get process id of swtpm on Ubuntu MATE Kinetic 22.10 #770

Closed TheJags closed 1 year ago

TheJags commented 1 year ago

Describe the bug: Cannot run/open a virtual machine.

When I try to run a virtual machine (which previously used to run just fine), I'm getting this error:

Error starting domain: internal error: Could not get process id of swtpm

Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 72, in cb_wrapper
    callback(asyncjob, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 108, in tmpcb
    callback(*args, **kwargs)
  File "/usr/share/virt-manager/virtManager/object/libvirtobject.py", line 57, in newfn
    ret = fn(self, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/object/domain.py", line 1402, in startup
    self._backend.create()
  File "/usr/lib/python3/dist-packages/libvirt.py", line 1352, in create
    raise libvirtError('virDomainCreate() failed')

libvirt.libvirtError: internal error: Could not get process id of swtpm

Required: To Reproduce (without these steps your issue may be deleted)

Steps to reproduce:

  1. Open Virtual Machine Manager
  2. Select previously created virtual machine
  3. Right-click on a vm and click, run
  4. Getting the error

Expected behavior: Expected behavior would be a virtual machine running inside a console window.

Desktop:

Versions of relevant components:

- Virtual Machine Details:

TPM Device: Type: Emulated

Advanced options: Model: CRB Version: 2.0

Log files: Please attach any log files. If using a VM and it was started with libvirt, attach the logfile found in /var/log/swtpm/libvirt/qemu/VM-NAME-swtpm.log.

(1) /var/log/swtpm/libvirt/qemu/jammy-swtpm.log

Starting vTPM manufacturing as tss:tss @ Tue 25 Jan 2022 06:48:26 PM
Successfully created RSA 2048 EK with handle 0x81010001.

  Invoking /usr/share/swtpm/swtpm-localca --type ek --ek 83043855c481eed0c2a8cf35a802f93ae1349dcd00dc01a665f109780a158a3484809e00acfc78d3fefad75b6cc405d3d363d01d288a39335913290c049a53db9ade35ec14e068d3a01963145a9a55636731d38953adae4e9e8a4ed4659e5195c067c64e16705fed92619f5de775f33c34ddf5c568a40c7963cbdd4173234a61b9f3e83129fbc1df7b85b185f43ae857e83112c3c3741e978717c680d9691be88f2b83ced95c06f0485f71fd440f5dff858ab3fd04c7d8fa4e4c1f2dddecb39f39dbe9bc10d2d51bef9fb50a3f277e2d1e1df06c07119af1bf7760fb2663d7f940c02dc49bfaf90402e68a23a3fdee40ba57ce6fd776378090842c813e7293d9 --dir /tmp/swtpm_setup.certs.CBA2F1 --logfile /var/log/swtpm/libvirt/qemu/jammy-swtpm.log --vmid jammy:c857b1ce-7157-4be5-ae36-29cf22a62ec8 --tpm-spec-family 2.0 --tpm-spec-level 0 --tpm-spec-revision 164 --tpm-manufacturer id:00001014 --tpm-model swtpm --tpm-version id:20191023 --tpm2 --configfile /etc/swtpm-localca.conf --optsfile /etc/swtpm-localca.options
Successfully created EK certificate locally.

  Invoking /usr/share/swtpm/swtpm-localca --type platform --ek 83043855c481eed0c2a8cf35a802f93ae1349dcd00dc01a665f109780a158a3484809e00acfc78d3fefad75b6cc405d3d363d01d288a39335913290c049a53db9ade35ec14e068d3a01963145a9a55636731d38953adae4e9e8a4ed4659e5195c067c64e16705fed92619f5de775f33c34ddf5c568a40c7963cbdd4173234a61b9f3e83129fbc1df7b85b185f43ae857e83112c3c3741e978717c680d9691be88f2b83ced95c06f0485f71fd440f5dff858ab3fd04c7d8fa4e4c1f2dddecb39f39dbe9bc10d2d51bef9fb50a3f277e2d1e1df06c07119af1bf7760fb2663d7f940c02dc49bfaf90402e68a23a3fdee40ba57ce6fd776378090842c813e7293d9 --dir /tmp/swtpm_setup.certs.CBA2F1 --logfile /var/log/swtpm/libvirt/qemu/jammy-swtpm.log --vmid jammy:c857b1ce-7157-4be5-ae36-29cf22a62ec8 --tpm-spec-family 2.0 --tpm-spec-level 0 --tpm-spec-revision 164 --tpm-manufacturer id:00001014 --tpm-model swtpm --tpm-version id:20191023 --tpm2 --configfile /etc/swtpm-localca.conf --optsfile /etc/swtpm-localca.options

Successfully created platform certificate locally.
Successfully created NVRAM area 0x1c00002 for RSA 2048 EK certificate.
Successfully created NVRAM area 0x1c08000 for platform certificate.
Successfully created ECC EK with handle 0x81010016.

  Invoking /usr/share/swtpm/swtpm-localca --type ek --ek x=0bcedc5e24dc2d121efb2da97609be4a2b42cb0de8e267d149a6f01c716c12b4365d26f862ceb7922be012a232b2d784,y=b01e90b78d56b24a5dd1f0023c0f6fc663c0129cd33e22a3b21193726c0b68b27cd0e6dee4b4e4e3bab7620c76e307f1,id=secp384r1 --dir /tmp/swtpm_setup.certs.CBA2F1 --logfile /var/log/swtpm/libvirt/qemu/jammy-swtpm.log --vmid jammy:c857b1ce-7157-4be5-ae36-29cf22a62ec8 --tpm-spec-family 2.0 --tpm-spec-level 0 --tpm-spec-revision 164 --tpm-manufacturer id:00001014 --tpm-model swtpm --tpm-version id:20191023 --tpm2 --configfile /etc/swtpm-localca.conf --optsfile /etc/swtpm-localca.options

Successfully created EK certificate locally.
Successfully created NVRAM area 0x1c00016 for ECC EK certificate.
Successfully activated PCR banks sha256 among sha1,sha256,sha384,sha512.
Successfully authored TPM state.

Ending vTPM manufacturing @ Tue 25 Jan 2022 06:48:27 PM

Additional context: (2) /var/log/libvirt/qemu/jammy.log

2022-11-17 18:45:47.874+0000: starting up libvirt version: 8.6.0, package: 0ubuntu3 (Christian Ehrhardt <christian.ehrhardt@canonical.com> Tue, 04 Oct 2022 08:29:46 +0200), qemu version: 7.0.0Debian 1:7.0+dfsg-7ubuntu2, kernel: 6.0.9-060009-generic, hostname: um.exp.lab

LC_ALL=C \
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
HOME=/var/lib/libvirt/qemu/domain-4-jammy \
XDG_DATA_HOME=/var/lib/libvirt/qemu/domain-4-jammy/.local/share \
XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain-4-jammy/.cache \
XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain-4-jammy/.config \
/usr/bin/qemu-system-x86_64 \
-name guest=jammy,debug-threads=on \
-S \
-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain-4-jammy/master-key.aes"}' \
-machine pc-q35-6.0,usb=off,vmport=off,dump-guest-core=off,memory-backend=pc.ram \
-accel kvm \
-cpu Westmere-IBRS,vme=on,pclmulqdq=on,vmx=on,pdcm=on,pcid=on,x2apic=on,tsc-deadline=on,hypervisor=on,arat=on,tsc-adjust=on,umip=on,stibp=on,arch-capabilities=on,ssbd=on,pdpe1gb=on,rdtscp=on,ibpb=on,ibrs=on,amd-stibp=on,amd-ssbd=on,skip-l1dfl-vmentry=on,pschange-mc-no=on \
-m 4096 \
-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":4294967296}' \
-overcommit mem-lock=off \
-smp 2,sockets=2,cores=1,threads=1 \
-uuid c857b1ce-7157-4be5-ae36-29cf22a62ec8 \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,fd=32,server=on,wait=off \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=utc,driftfix=slew \
-global kvm-pit.lost_tick_policy=delay \
-no-hpet \
-no-shutdown \
-global ICH9-LPC.disable_s3=1 \
-global ICH9-LPC.disable_s4=1 \
-boot strict=on \
-device '{"driver":"pcie-root-port","port":16,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x2"}' \
-device '{"driver":"pcie-root-port","port":17,"chassis":2,"id":"pci.2","bus":"pcie.0","addr":"0x2.0x1"}' \
-device '{"driver":"pcie-root-port","port":18,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x2.0x2"}' \
-device '{"driver":"pcie-root-port","port":19,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x2.0x3"}' \
-device '{"driver":"pcie-root-port","port":20,"chassis":5,"id":"pci.5","bus":"pcie.0","addr":"0x2.0x4"}' \
-device '{"driver":"pcie-root-port","port":21,"chassis":6,"id":"pci.6","bus":"pcie.0","addr":"0x2.0x5"}' \
-device '{"driver":"pcie-root-port","port":22,"chassis":7,"id":"pci.7","bus":"pcie.0","addr":"0x2.0x6"}' \
-device '{"driver":"pcie-root-port","port":23,"chassis":8,"id":"pci.8","bus":"pcie.0","addr":"0x2.0x7"}' \
-device '{"driver":"pcie-pci-bridge","id":"pci.9","bus":"pci.1","addr":"0x0"}' \
-device '{"driver":"pcie-root-port","port":24,"chassis":10,"id":"pci.10","bus":"pcie.0","addr":"0x3"}' \
-device '{"driver":"qemu-xhci","p2":15,"p3":15,"id":"usb","bus":"pci.4","addr":"0x0"}' \
-device '{"driver":"virtio-serial-pci","id":"virtio-serial0","bus":"pci.5","addr":"0x0"}' \
-blockdev '{"driver":"file","filename":"/media/admn/310GB/QEMU_KVM/jammy.qcow2","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-2-format","read-only":false,"driver":"qcow2","file":"libvirt-2-storage","backing":null}' \
-device '{"driver":"virtio-blk-pci","bus":"pci.6","addr":"0x0","drive":"libvirt-2-format","id":"virtio-disk0","bootindex":1}' \
-blockdev '{"driver":"file","filename":"/media/admn/1-6TB/Softwares/Linux/ubuntu-mate-2022.04.30-desktop-amd64.iso","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-1-format","read-only":true,"driver":"raw","file":"libvirt-1-storage"}' \
-device '{"driver":"ide-cd","bus":"ide.0","drive":"libvirt-1-format","id":"sata0-0-0","bootindex":2}' \
-fsdev local,security_model=mapped,id=fsdev-fs0,path=/media/admn/1-6TB/QEMU_Shared \
-device '{"driver":"virtio-9p-pci","id":"fs0","fsdev":"fsdev-fs0","mount_tag":"/shared","bus":"pci.2","addr":"0x0"}' \
-netdev tap,fd=33,vhost=on,vhostfd=35,id=hostnet0 \
-device '{"driver":"virtio-net-pci","netdev":"hostnet0","id":"net0","mac":"52:54:00:9a:a5:93","bus":"pci.3","addr":"0x0"}' \
-chardev pty,id=charserial0 \
-device '{"driver":"isa-serial","chardev":"charserial0","id":"serial0","index":0}' \
-chardev socket,id=charchannel0,fd=31,server=on,wait=off \
-device '{"driver":"virtserialport","bus":"virtio-serial0.0","nr":1,"chardev":"charchannel0","id":"channel0","name":"org.qemu.guest_agent.0"}' \
-chardev spicevmc,id=charchannel1,name=vdagent \
-device '{"driver":"virtserialport","bus":"virtio-serial0.0","nr":2,"chardev":"charchannel1","id":"channel1","name":"com.redhat.spice.0"}' \
-chardev socket,id=chrtpm,path=/run/libvirt/qemu/swtpm/4-jammy-swtpm.sock \
-tpmdev emulator,id=tpm-tpm0,chardev=chrtpm \
-device '{"driver":"tpm-crb","tpmdev":"tpm-tpm0","id":"tpm0"}' \
-device '{"driver":"usb-tablet","id":"input0","bus":"usb.0","port":"1"}' \
-audiodev '{"id":"audio1","driver":"spice"}' \
-spice port=5900,addr=127.0.0.1,disable-ticketing=on,image-compression=off,seamless-migration=on \
-device '{"driver":"virtio-vga","id":"video0","max_outputs":1,"bus":"pcie.0","addr":"0x1"}' \
-device '{"driver":"ich9-intel-hda","id":"sound0","bus":"pcie.0","addr":"0x1b"}' \
-device '{"driver":"hda-duplex","id":"sound0-codec0","bus":"sound0.0","cad":0,"audiodev":"audio1"}' \
-device '{"driver":"i6300esb","id":"watchdog0","bus":"pci.9","addr":"0x1"}' \
-watchdog-action reset \
-chardev spicevmc,id=charredir0,name=usbredir \
-device '{"driver":"usb-redir","chardev":"charredir0","id":"redir0","bus":"usb.0","port":"2"}' \
-chardev spicevmc,id=charredir1,name=usbredir \
-device '{"driver":"usb-redir","chardev":"charredir1","id":"redir1","bus":"usb.0","port":"3"}' \
-device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.7","addr":"0x0"}' \
-object '{"qom-type":"rng-random","id":"objrng0","filename":"/dev/urandom"}' \
-device '{"driver":"virtio-rng-pci","rng":"objrng0","id":"rng0","bus":"pci.8","addr":"0x0"}' \
-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on

libvirt:  error : libvirtd quit during handshake: Input/output error

2022-11-17 18:45:47.972+0000: shutting down, reason=failed

(3) Guest OS XML (Ubuntu MATE Kinetic 22.10):

<domain type="kvm">
  <name>jammy</name>
  <uuid>c857b1ce-7157-4be5-ae36-29cf22a62ec8</uuid>
  <metadata>
    <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
      <libosinfo:os id="http://ubuntu.com/ubuntu/21.04"/>
    </libosinfo:libosinfo>
  </metadata>
  <memory unit="KiB">4194304</memory>
  <currentMemory unit="KiB">4194304</currentMemory>
  <vcpu placement="static">2</vcpu>
  <os>
    <type arch="x86_64" machine="pc-q35-6.0">hvm</type>
  </os>
  <features>
    <acpi/>
    <apic/>
    <vmport state="off"/>
  </features>
  <cpu mode="host-model" check="partial"/>
  <clock offset="utc">
    <timer name="rtc" tickpolicy="catchup"/>
    <timer name="pit" tickpolicy="delay"/>
    <timer name="hpet" present="no"/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <pm>
    <suspend-to-mem enabled="no"/>
    <suspend-to-disk enabled="no"/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type="file" device="disk">
      <driver name="qemu" type="qcow2"/>
      <source file="/media/admn/310GB/QEMU_KVM/jammy.qcow2"/>
      <target dev="vda" bus="virtio"/>
      <boot order="1"/>
      <address type="pci" domain="0x0000" bus="0x06" slot="0x00" function="0x0"/>
    </disk>
    <disk type="file" device="cdrom">
      <driver name="qemu" type="raw"/>
      <source file="/media/admn/1-6TB/Softwares/Linux/ubuntu-mate-2022.04.30-desktop-amd64.iso"/>
      <target dev="sda" bus="sata"/>
      <readonly/>
      <boot order="2"/>
      <address type="drive" controller="0" bus="0" target="0" unit="0"/>
    </disk>
    <controller type="usb" index="0" model="qemu-xhci" ports="15">
      <address type="pci" domain="0x0000" bus="0x04" slot="0x00" function="0x0"/>
    </controller>
    <controller type="sata" index="0">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x1f" function="0x2"/>
    </controller>
    <controller type="pci" index="0" model="pcie-root"/>
    <controller type="pci" index="1" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="1" port="0x10"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x0" multifunction="on"/>
    </controller>
    <controller type="pci" index="2" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="2" port="0x11"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x1"/>
    </controller>
    <controller type="pci" index="3" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="3" port="0x12"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x2"/>
    </controller>
    <controller type="pci" index="4" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="4" port="0x13"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x3"/>
    </controller>
    <controller type="pci" index="5" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="5" port="0x14"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x4"/>
    </controller>
    <controller type="pci" index="6" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="6" port="0x15"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x5"/>
    </controller>
    <controller type="pci" index="7" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="7" port="0x16"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x6"/>
    </controller>
    <controller type="pci" index="8" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="8" port="0x17"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x7"/>
    </controller>
    <controller type="pci" index="9" model="pcie-to-pci-bridge">
      <model name="pcie-pci-bridge"/>
      <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0"/>
    </controller>
    <controller type="pci" index="10" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="10" port="0x18"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x0"/>
    </controller>
    <controller type="virtio-serial" index="0">
      <address type="pci" domain="0x0000" bus="0x05" slot="0x00" function="0x0"/>
    </controller>
    <filesystem type="mount" accessmode="mapped">
      <source dir="/media/admn/1-6TB/QEMU_Shared"/>
      <target dir="/shared"/>
      <address type="pci" domain="0x0000" bus="0x02" slot="0x00" function="0x0"/>
    </filesystem>
    <interface type="direct">
      <mac address="52:54:00:9a:a5:93"/>
      <source dev="wlxd0374547816a" mode="bridge"/>
      <model type="virtio"/>
      <link state="up"/>
      <address type="pci" domain="0x0000" bus="0x03" slot="0x00" function="0x0"/>
    </interface>
    <serial type="pty">
      <target type="isa-serial" port="0">
        <model name="isa-serial"/>
      </target>
    </serial>
    <console type="pty">
      <target type="serial" port="0"/>
    </console>
    <channel type="unix">
      <target type="virtio" name="org.qemu.guest_agent.0"/>
      <address type="virtio-serial" controller="0" bus="0" port="1"/>
    </channel>
    <channel type="spicevmc">
      <target type="virtio" name="com.redhat.spice.0"/>
      <address type="virtio-serial" controller="0" bus="0" port="2"/>
    </channel>
    <input type="tablet" bus="usb">
      <address type="usb" bus="0" port="1"/>
    </input>
    <input type="mouse" bus="ps2"/>
    <input type="keyboard" bus="ps2"/>
    <tpm model="tpm-crb">
      <backend type="emulator" version="2.0"/>
    </tpm>
    <graphics type="spice" autoport="yes">
      <listen type="address"/>
      <image compression="off"/>
    </graphics>
    <sound model="ich9">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x1b" function="0x0"/>
    </sound>
    <audio id="1" type="spice"/>
    <video>
      <model type="virtio" heads="1" primary="yes"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x0"/>
    </video>
    <redirdev bus="usb" type="spicevmc">
      <address type="usb" bus="0" port="2"/>
    </redirdev>
    <redirdev bus="usb" type="spicevmc">
      <address type="usb" bus="0" port="3"/>
    </redirdev>
    <watchdog model="i6300esb" action="reset">
      <address type="pci" domain="0x0000" bus="0x09" slot="0x01" function="0x0"/>
    </watchdog>
    <memballoon model="virtio">
      <address type="pci" domain="0x0000" bus="0x07" slot="0x00" function="0x0"/>
    </memballoon>
    <rng model="virtio">
      <backend model="random">/dev/urandom</backend>
      <address type="pci" domain="0x0000" bus="0x08" slot="0x00" function="0x0"/>
    </rng>
  </devices>
</domain>
stefanberger commented 1 year ago

If AppArmor is enabled on your Ubuntu system, can you please try to disable it entirely and try again.

stefanberger commented 1 year ago

Otherwise, can you please check the audit log in /var/log/audit/audit.log for potential failures?

TheJags commented 1 year ago

Otherwise, can you please check the audit log in /var/log/audit/audit.log for potential failures?

@stefanberger Thank you so much for quick replies.

(1) Somehow there is neither /var/log/audit/ directory nor audit.log inside /var/log/ at all.

(2) Though disabling AppArmor entirely did the trick.

sudo systemctl disable apparmor

Reboot

And the virtual machine run/open just fine. Tried again after enabling AppArmor and it did not run.

So, is it advisable to (a) disable AppArmor for swtpm or (b) edit AppArmor profile for swtpm?

(a) Disable AppArmor for swtpm:

Note: I have not tried this yet; just modified the commands found on this page:

https://linuxconfig.org/how-to-disable-apparmor-on-ubuntu-20-04-focal-fossa-linux

sudo ln -s /etc/apparmor.d/usr.bin.swtpm /etc/apparmor.d/disable/

apparmor_parser -R /etc/apparmor.d/disable/usr.bin.swtpm

(b) Edit AppArmor profile for swtpm:

sudo nano /etc/apparmor.d/usr.bin.swtpm

 1 # vim:syntax=apparmor
 2 # AppArmor policy for swtpm
 3
 4 #include <tunables/global>
 5
 6 profile swtpm /usr/bin/swtpm {
 7   #include <abstractions/base>
 8   #include <abstractions/openssl>
 9
10   # Site-specific additions and overrides. See local/README for details.
11   #include <local/usr.bin.swtpm>
12
13   capability chown,
14   capability dac_override,
15   capability dac_read_search,
16   capability fowner,
17   capability fsetid,
18   capability setgid,
19   capability setuid,
20
21   network inet stream,
22   network inet6 stream,
23   unix (send) type=dgram addr=none peer=(addr=none),
24   unix (send, receive) type=stream addr=none peer=(label=libvirt-*),
25
26   /usr/bin/swtpm rm,
27
28   /tmp/** rwk,
29   owner @{HOME}/** rwk,
30   owner /var/lib/libvirt/swtpm/** rwk,
31   /run/libvirt/qemu/swtpm/*.sock rwk,
32   owner /var/log/swtpm/libvirt/qemu/*.log rwk,
33   owner /run/libvirt/qemu/swtpm/*.pid rwk,
34   owner /dev/vtpmx rw,
35   owner /etc/nsswitch.conf r,
36   owner /var/lib/swtpm/** rwk,
37   owner /run/swtpm/sock rw,
38 }
39
TheJags commented 1 year ago 
$ sudo apparmor_status

apparmor module is loaded.
35 profiles are loaded.
33 profiles are in enforce mode.

   /usr/bin/man
   /usr/bin/redshift
   /usr/lib/NetworkManager/nm-dhcp-client.action
   /usr/lib/NetworkManager/nm-dhcp-helper
   /usr/lib/connman/scripts/dhclient-script
   /usr/lib/cups/backend/cups-pdf
   /usr/lib/ipsec/charon
   /usr/lib/ipsec/lookip
   /usr/lib/ipsec/stroke
   /usr/lib/lightdm/lightdm-guest-session
   /usr/lib/lightdm/lightdm-guest-session//chromium
   /usr/sbin/cups-browsed
   /usr/sbin/cupsd
   /usr/sbin/cupsd//third_party
   /usr/sbin/gpsd
   /usr/sbin/ntpd
   /usr/sbin/unbound
   /{,usr/}sbin/dhclient
   ippusbxd
   libreoffice-senddoc
   libreoffice-soffice//gpg
   libreoffice-xpdfimport
   libvirt-c857b1ce-7157-4be5-ae36-29cf22a62ec8
   libvirtd
   libvirtd//qemu_bridge_helper
   lsb_release
   man_filter
   man_groff
   nvidia_modprobe
   nvidia_modprobe//kmod
   swtpm
   tcpdump
   virt-aa-helper

2 profiles are in complain mode.
   libreoffice-oosplash
   libreoffice-soffice

0 profiles are in kill mode.
0 profiles are in unconfined mode.
7 processes have profiles defined.

7 processes are in enforce mode.
   /usr/bin/redshift (2013) 
   /usr/sbin/cups-browsed (2289) 
   /usr/sbin/cupsd (1347) 
   /usr/lib/cups/notifier/dbus (1386) /usr/sbin/cupsd
   /usr/sbin/ntpd (2436) 
   /usr/sbin/unbound (1383) 
   /usr/sbin/libvirtd (1349) libvirtd

0 processes are in complain mode.
0 processes are unconfined but have a profile defined.
0 processes are in mixed mode.
0 processes are in kill mode.
$
stefanberger commented 1 year ago

Thanks for trying this out. At least we know what the problem is now.

Did you take my swtpm package (frpm my PPA) or is this from Ubuntu or built from repo?

It's quite possible that a recent extension to swtpm now needs additional AppArmor rules either in swtpm or libvirt -- possibly the latter, but I am not sure.

You would have to enable auditing on your system to see the failure that causes the abort of swtpm. That would help fix the profile.

TheJags commented 1 year ago

Many thanks.

(1) I have installed swtpm from your PPA:

https://launchpad.net/~stefanberger/+archive/ubuntu/swtpm-jammy

(2) The same VM used to run just fine with swtpm, around 2 weeks ago, so as you mentioned, "a recent extension to swtpm now needs additional AppArmor rules either in swtpm or libvirt" or recent Ubuntu updates must have changed in with AppArmor/Ubuntu.

(3) I've installed auditd

sudo apt install auditd audispd-plugins

(4) Tried to run the VM again (it did not run as AppArmor is enabled), and here are entries from /var/log/audit/audit.log:

type=AVC msg=audit(1668718959.256:68): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="libvirt-c857b1ce-7157-4be5-ae36-29cf22a62ec8" pid=10085 comm="apparmor_parser"

type=SYSCALL msg=audit(1668718959.256:68): arch=c000003e syscall=1 success=yes exit=57153 a0=5 a1=55bd38e2fc50 a2=df41 a3=0 items=0 ppid=10084 pid=10085 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="apparmor_parser" exe="/usr/sbin/apparmor_parser" subj=unconfined key=(null)ARCH=x86_64 SYSCALL=write AUID="unset" UID="root" GID="root" EUID="root" SUID="root" FSUID="root" EGID="root" SGID="root" FSGID="root"

type=PROCTITLE msg=audit(1668718959.256:68): proctitle=2F7362696E2F61707061726D6F725F706172736572002D72002F6574632F61707061726D6F722E642F6C6962766972742F6C6962766972742D63383537623163652D373135372D346265352D616533362D323963663232613632656338

type=VIRT_MACHINE_ID msg=audit(1668718959.264:69): pid=1349 uid=0 auid=4294967295 ses=4294967295 subj=libvirtd msg='virt=kvm vm="jammy" uuid=c857b1ce-7157-4be5-ae36-29cf22a62ec8 vm-ctx=libvirt-c857b1ce-7157-4be5-ae36-29cf22a62ec8 img-ctx=libvirt-c857b1ce-7157-4be5-ae36-29cf22a62ec8 model=apparmor exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"

type=VIRT_MACHINE_ID msg=audit(1668718959.264:70): pid=1349 uid=0 auid=4294967295 ses=4294967295 subj=libvirtd msg='virt=kvm vm="jammy" uuid=c857b1ce-7157-4be5-ae36-29cf22a62ec8 vm-ctx=+64055:+108 img-ctx=+64055:+108 model=dac exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"

type=AVC msg=audit(1668718959.492:71): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="libvirt-c857b1ce-7157-4be5-ae36-29cf22a62ec8" pid=10088 comm="apparmor_parser"

type=SYSCALL msg=audit(1668718959.492:71): arch=c000003e syscall=1 success=yes exit=57177 a0=5 a1=5574c9efe100 a2=df59 a3=0 items=0 ppid=10087 pid=10088 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="apparmor_parser" exe="/usr/sbin/apparmor_parser" subj=unconfined key=(null)ARCH=x86_64 SYSCALL=write AUID="unset" UID="root" GID="root" EUID="root" SUID="root" FSUID="root" EGID="root" SGID="root" FSGID="root"

type=PROCTITLE msg=audit(1668718959.492:71): proctitle=2F7362696E2F61707061726D6F725F706172736572002D72002F6574632F61707061726D6F722E642F6C6962766972742F6C6962766972742D63383537623163652D373135372D346265352D616533362D323963663232613632656338

type=AVC msg=audit(1668718959.720:72): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="libvirt-c857b1ce-7157-4be5-ae36-29cf22a62ec8" pid=10092 comm="apparmor_parser"

type=SYSCALL msg=audit(1668718959.720:72): arch=c000003e syscall=1 success=yes exit=57577 a0=5 a1=55e9da2dc530 a2=e0e9 a3=0 items=0 ppid=10091 pid=10092 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="apparmor_parser" exe="/usr/sbin/apparmor_parser" subj=unconfined key=(null)ARCH=x86_64 SYSCALL=write AUID="unset" UID="root" GID="root" EUID="root" SUID="root" FSUID="root" EGID="root" SGID="root" FSGID="root"

type=PROCTITLE msg=audit(1668718959.720:72): proctitle=2F7362696E2F61707061726D6F725F706172736572002D72002F6574632F61707061726D6F722E642F6C6962766972742F6C6962766972742D63383537623163652D373135372D346265352D616533362D323963663232613632656338

type=AVC msg=audit(1668718959.956:73): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="libvirt-c857b1ce-7157-4be5-ae36-29cf22a62ec8" pid=10096 comm="apparmor_parser"

type=SYSCALL msg=audit(1668718959.956:73): arch=c000003e syscall=1 success=yes exit=57577 a0=5 a1=55d867805990 a2=e0e9 a3=0 items=0 ppid=10095 pid=10096 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="apparmor_parser" exe="/usr/sbin/apparmor_parser" subj=unconfined key=(null)ARCH=x86_64 SYSCALL=write AUID="unset" UID="root" GID="root" EUID="root" SUID="root" FSUID="root" EGID="root" SGID="root" FSGID="root"

type=PROCTITLE msg=audit(1668718959.956:73): proctitle=2F7362696E2F61707061726D6F725F706172736572002D72002F6574632F61707061726D6F722E642F6C6962766972742F6C6962766972742D63383537623163652D373135372D346265352D616533362D323963663232613632656338

type=AVC msg=audit(1668718959.980:74): apparmor="DENIED" operation="file_inherit" profile="swtpm" name="/run/libvirt/qemu/swtpm/2-jammy-swtpm.pid" pid=10102 comm="swtpm" requested_mask="w" denied_mask="w" fsuid=135 ouid=0FSUID="swtpm" OUID="root"

type=SYSCALL msg=audit(1668718959.980:74): arch=c000003e syscall=59 success=yes exit=0 a0=7fcc5c065710 a1=7fcc5c06ab90 a2=7ffe9b4d0828 a3=0 items=0 ppid=1 pid=10102 auid=4294967295 uid=135 gid=147 euid=135 suid=135 fsuid=135 egid=147 sgid=147 fsgid=147 tty=(none) ses=4294967295 comm="swtpm" exe="/usr/bin/swtpm" subj=swtpm key=(null)ARCH=x86_64 SYSCALL=execve AUID="unset" UID="swtpm" GID="swtpm" EUID="swtpm" SUID="swtpm" FSUID="swtpm" EGID="swtpm" SGID="swtpm" FSGID="swtpm"

type=EXECVE msg=audit(1668718959.980:74): argc=10 a0="/usr/bin/swtpm" a1="socket" a2="--ctrl" a3="type=unixio,path=/run/libvirt/qemu/swtpm/2-jammy-swtpm.sock,mode=0600" a4="--tpmstate" a5="dir=/var/lib/libvirt/swtpm/c857b1ce-7157-4be5-ae36-29cf22a62ec8/tpm2,mode=0600" a6="--log" a7="file=/var/log/swtpm/libvirt/qemu/jammy-swtpm.log" a8="--terminate" a9="--tpm2"

type=PROCTITLE msg=audit(1668718959.980:74): proctitle=2F7573722F62696E2F737774706D00736F636B6574002D2D6374726C00747970653D756E6978696F2C706174683D2F72756E2F6C6962766972742F71656D752F737774706D2F322D6A616D6D792D737774706D2E736F636B2C6D6F64653D30363030002D2D74706D7374617465006469723D2F7661722F6C69622F6C69627669

type=VIRT_RESOURCE msg=audit(1668718960.012:75): pid=1349 uid=0 auid=4294967295 ses=4294967295 subj=libvirtd msg='virt=kvm resrc=net reason=open vm="jammy" uuid=c857b1ce-7157-4be5-ae36-29cf22a62ec8 net=52:54:00:9a:a5:93 path="macvtap1" rdev=? exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"

type=ANOM_PROMISCUOUS msg=audit(1668718960.028:76): dev=wlxd0374547816a prom=256 old_prom=0 auid=4294967295 uid=0 gid=0 ses=4294967295AUID="unset" UID="root" GID="root"

type=AVC msg=audit(1668718960.236:77): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="libvirt-c857b1ce-7157-4be5-ae36-29cf22a62ec8" pid=10122 comm="apparmor_parser"

type=SYSCALL msg=audit(1668718960.236:77): arch=c000003e syscall=1 success=yes exit=57609 a0=5 a1=5559a97017f0 a2=e109 a3=0 items=0 ppid=10121 pid=10122 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="apparmor_parser" exe="/usr/sbin/apparmor_parser" subj=unconfined key=(null)ARCH=x86_64 SYSCALL=write AUID="unset" UID="root" GID="root" EUID="root" SUID="root" FSUID="root" EGID="root" SGID="root" FSGID="root"

type=PROCTITLE msg=audit(1668718960.236:77): proctitle=2F7362696E2F61707061726D6F725F706172736572002D72002F6574632F61707061726D6F722E642F6C6962766972742F6C6962766972742D63383537623163652D373135372D346265352D616533362D323963663232613632656338

type=VIRT_RESOURCE msg=audit(1668718960.240:78): pid=1349 uid=0 auid=4294967295 ses=4294967295 subj=libvirtd msg='virt=kvm resrc=net reason=open vm="jammy" uuid=c857b1ce-7157-4be5-ae36-29cf22a62ec8 net=52:54:00:9a:a5:93 path="/dev/vhost-net" rdev=0A:EE exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"

type=BPF msg=audit(1668718960.364:79): prog-id=64 op=LOAD

type=VIRT_RESOURCE msg=audit(1668718960.364:80): pid=1349 uid=0 auid=4294967295 ses=4294967295 subj=libvirtd msg='virt=kvm resrc=cgroup reason=deny vm="jammy" uuid=c857b1ce-7157-4be5-ae36-29cf22a62ec8 cgroup="/sys/fs/cgroup/machine.slice/machine-qemu\x2d2\x2djammy.scope/" class=all exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"

type=VIRT_RESOURCE msg=audit(1668718960.364:81): pid=1349 uid=0 auid=4294967295 ses=4294967295 subj=libvirtd msg='virt=kvm resrc=cgroup reason=allow vm="jammy" uuid=c857b1ce-7157-4be5-ae36-29cf22a62ec8 cgroup="/sys/fs/cgroup/machine.slice/machine-qemu\x2d2\x2djammy.scope/" class=path path="/dev/null" rdev=01:03 acl=rw exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"

type=VIRT_RESOURCE msg=audit(1668718960.364:82): pid=1349 uid=0 auid=4294967295 ses=4294967295 subj=libvirtd msg='virt=kvm resrc=cgroup reason=allow vm="jammy" uuid=c857b1ce-7157-4be5-ae36-29cf22a62ec8 cgroup="/sys/fs/cgroup/machine.slice/machine-qemu\x2d2\x2djammy.scope/" class=path path="/dev/full" rdev=01:07 acl=rw exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"

type=VIRT_RESOURCE msg=audit(1668718960.364:83): pid=1349 uid=0 auid=4294967295 ses=4294967295 subj=libvirtd msg='virt=kvm resrc=cgroup reason=allow vm="jammy" uuid=c857b1ce-7157-4be5-ae36-29cf22a62ec8 cgroup="/sys/fs/cgroup/machine.slice/machine-qemu\x2d2\x2djammy.scope/" class=path path="/dev/zero" rdev=01:05 acl=rw exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"

type=VIRT_RESOURCE msg=audit(1668718960.364:84): pid=1349 uid=0 auid=4294967295 ses=4294967295 subj=libvirtd msg='virt=kvm resrc=cgroup reason=allow vm="jammy" uuid=c857b1ce-7157-4be5-ae36-29cf22a62ec8 cgroup="/sys/fs/cgroup/machine.slice/machine-qemu\x2d2\x2djammy.scope/" class=path path="/dev/random" rdev=01:08 acl=rw exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"

type=VIRT_RESOURCE msg=audit(1668718960.364:85): pid=1349 uid=0 auid=4294967295 ses=4294967295 subj=libvirtd msg='virt=kvm resrc=cgroup reason=allow vm="jammy" uuid=c857b1ce-7157-4be5-ae36-29cf22a62ec8 cgroup="/sys/fs/cgroup/machine.slice/machine-qemu\x2d2\x2djammy.scope/" class=path path="/dev/urandom" rdev=01:09 acl=rw exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"

type=VIRT_RESOURCE msg=audit(1668718960.364:86): pid=1349 uid=0 auid=4294967295 ses=4294967295 subj=libvirtd msg='virt=kvm resrc=cgroup reason=allow vm="jammy" uuid=c857b1ce-7157-4be5-ae36-29cf22a62ec8 cgroup="/sys/fs/cgroup/machine.slice/machine-qemu\x2d2\x2djammy.scope/" class=path path="/dev/ptmx" rdev=05:02 acl=rw exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"

type=VIRT_RESOURCE msg=audit(1668718960.364:87): pid=1349 uid=0 auid=4294967295 ses=4294967295 subj=libvirtd msg='virt=kvm resrc=cgroup reason=allow vm="jammy" uuid=c857b1ce-7157-4be5-ae36-29cf22a62ec8 cgroup="/sys/fs/cgroup/machine.slice/machine-qemu\x2d2\x2djammy.scope/" class=path path="/dev/kvm" rdev=0A:E8 acl=rw exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"

type=VIRT_RESOURCE msg=audit(1668718960.364:88): pid=1349 uid=0 auid=4294967295 ses=4294967295 subj=libvirtd msg='virt=kvm resrc=cgroup reason=allow vm="jammy" uuid=c857b1ce-7157-4be5-ae36-29cf22a62ec8 cgroup="/sys/fs/cgroup/machine.slice/machine-qemu\x2d2\x2djammy.scope/" class=major category=pty maj=88 acl=rw exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"

type=VIRT_RESOURCE msg=audit(1668718960.364:89): pid=1349 uid=0 auid=4294967295 ses=4294967295 subj=libvirtd msg='virt=kvm resrc=cgroup reason=allow vm="jammy" uuid=c857b1ce-7157-4be5-ae36-29cf22a62ec8 cgroup="/sys/fs/cgroup/machine.slice/machine-qemu\x2d2\x2djammy.scope/" class=path path="/dev/urandom" rdev=01:09 acl=rw exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"

type=ANOM_PROMISCUOUS msg=audit(1668718960.576:90): dev=wlxd0374547816a prom=0 old_prom=256 auid=4294967295 uid=0 gid=0 ses=4294967295AUID="unset" UID="root" GID="root"

type=BPF msg=audit(1668718960.636:91): prog-id=0 op=UNLOAD

type=VIRT_RESOURCE msg=audit(1668718960.636:92): pid=1349 uid=0 auid=4294967295 ses=4294967295 subj=libvirtd msg='virt=kvm resrc=disk reason=start vm="jammy" uuid=c857b1ce-7157-4be5-ae36-29cf22a62ec8 old-disk="?" new-disk="/media/admn/310GB/QEMU_KVM/jammy.qcow2" exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"

type=VIRT_RESOURCE msg=audit(1668718960.636:93): pid=1349 uid=0 auid=4294967295 ses=4294967295 subj=libvirtd msg='virt=kvm resrc=disk reason=start vm="jammy" uuid=c857b1ce-7157-4be5-ae36-29cf22a62ec8 old-disk="?" new-disk="/media/admn/1-6TB/Softwares/Linux/ubuntu-mate-2022.04.30-desktop-amd64.iso" exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"

type=VIRT_RESOURCE msg=audit(1668718960.636:94): pid=1349 uid=0 auid=4294967295 ses=4294967295 subj=libvirtd msg='virt=kvm resrc=fs reason=start vm="jammy" uuid=c857b1ce-7157-4be5-ae36-29cf22a62ec8 old-fs="?" new-fs="/media/admn/1-6TB/QEMU_Shared" exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"

type=VIRT_RESOURCE msg=audit(1668718960.636:95): pid=1349 uid=0 auid=4294967295 ses=4294967295 subj=libvirtd msg='virt=kvm resrc=net reason=start vm="jammy" uuid=c857b1ce-7157-4be5-ae36-29cf22a62ec8 old-net="?" new-net="52:54:00:9a:a5:93" exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"

type=VIRT_RESOURCE msg=audit(1668718960.636:96): pid=1349 uid=0 auid=4294967295 ses=4294967295 subj=libvirtd msg='virt=kvm resrc=dev reason=start vm="jammy" uuid=c857b1ce-7157-4be5-ae36-29cf22a62ec8 bus=usb device=555342207265646972646576 exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"

type=VIRT_RESOURCE msg=audit(1668718960.636:97): pid=1349 uid=0 auid=4294967295 ses=4294967295 subj=libvirtd msg='virt=kvm resrc=dev reason=start vm="jammy" uuid=c857b1ce-7157-4be5-ae36-29cf22a62ec8 bus=usb device=555342207265646972646576 exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"

type=VIRT_RESOURCE msg=audit(1668718960.636:98): pid=1349 uid=0 auid=4294967295 ses=4294967295 subj=libvirtd msg='virt=kvm resrc=rng reason=start vm="jammy" uuid=c857b1ce-7157-4be5-ae36-29cf22a62ec8 old-rng="?" new-rng="/dev/urandom" exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"

type=VIRT_RESOURCE msg=audit(1668718960.636:99): pid=1349 uid=0 auid=4294967295 ses=4294967295 subj=libvirtd msg='virt=kvm resrc=tpm-emulator reason=start vm="jammy" uuid=c857b1ce-7157-4be5-ae36-29cf22a62ec8 device="?" exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"

type=VIRT_RESOURCE msg=audit(1668718960.636:100): pid=1349 uid=0 auid=4294967295 ses=4294967295 subj=libvirtd msg='virt=kvm resrc=mem reason=start vm="jammy" uuid=c857b1ce-7157-4be5-ae36-29cf22a62ec8 old-mem=0 new-mem=4194304 exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"

type=VIRT_RESOURCE msg=audit(1668718960.636:101): pid=1349 uid=0 auid=4294967295 ses=4294967295 subj=libvirtd msg='virt=kvm resrc=vcpu reason=start vm="jammy" uuid=c857b1ce-7157-4be5-ae36-29cf22a62ec8 old-vcpu=0 new-vcpu=2 exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"

type=VIRT_CONTROL msg=audit(1668718960.636:102): pid=1349 uid=0 auid=4294967295 ses=4294967295 subj=libvirtd msg='virt=kvm op=start reason=booted vm="jammy" uuid=c857b1ce-7157-4be5-ae36-29cf22a62ec8 vm-pid=0 exe="/usr/sbin/libvirtd" hostname=? addr=? terminal=? res=failed'UID="root" AUID="unset"
TheJags commented 1 year ago

Is this the line we are looking for in audit.log?

type=AVC msg=audit(1668718959.980:74): apparmor="DENIED" operation="file_inherit" profile="swtpm" name="/run/libvirt/qemu/swtpm/2-jammy-swtpm.pid" pid=10102 comm="swtpm" requested_mask="w" denied_mask="w" fsuid=135 ouid=0FSUID="swtpm" OUID="root"
stefanberger commented 1 year ago

Is this the line we are looking for in audit.log?

Yes!

We have been writing pid-files for a long time so this is odd that it wouldn't allow to write to /run/libvirt/qemu/swtpm/2-jammy-swtpm.pid . The profile is swtpm. Hm, we have this here in our profile.

https://github.com/stefanberger/swtpm/blob/dcd1b575e67811570a27661b8ff53ef5316b9ccf/debian/usr.bin.swtpm#L33

From the denial we get: fsuid=135 ouid=0�FSUID="swtpm" OUID="root".

In the libvirt profile we have this here:

https://github.com/libvirt/libvirt/blob/0be7d0f1cb257260758e38ffafcef5d2e0b43b86/src/security/virt-aa-helper.c#L1240-L1243

            virBufferAsprintf(&buf,
                "  \"%s/libvirt/qemu/swtpm/%s-swtpm.pid\" rw,\n",
                RUNSTATEDIR, shortName);

It's quite possible that the 'owner' part in the swtpm profile gets into the way. @lvoytek

stefanberger commented 1 year ago

I am actually surprised that the swtpm profile becomes active when libvirt starts the VM rather than the custom libvirt profile being used. Hm.

stefanberger commented 1 year ago

My next suggestion would be to run the system with AppArmor in enforcing mode but put only the swtpm profile into complain mode:

sudo aa-complain /usr/bin/swtpm
lvoytek commented 1 year ago

Hello, This is an issue caused by libvirt where pid files in /run/libvirt/qemu/swtpm/ are being set with root permissions rather than swtpm user permissions as they did in Jammy. I'm currently working on libvirt to get this fixed in upstream and kinetic. The Ubuntu bug is located here. In the meantime adding the line

/run/libvirt/qemu/swtpm/*.pid rwk,

to /etc/apparmor.d/local/usr.bin.swtpm will fix the issue.

stefanberger commented 1 year ago

@lvoytek Thanks for letting us know!

This is an issue caused by libvirt where pid files in /run/libvirt/qemu/swtpm/ are being set with root permissions rather than swtpm user permissions as they did in Jammy.

swtpm writes its own pid file but cannot do it due to ... what? Ownership of the directory? Having to overwrite an old pid file? What has changed since Jammy?

I'm currently working on libvirt to get this fixed in upstream and kinetic. The Ubuntu bug is located here. In the meantime adding the line

Also good to know. Thanks.

TheJags commented 1 year ago

@lvoytek @stefanberger

Thank you so much.

Creating (the file was not there)

sudo nano /etc/apparmor.d/local/usr.bin.swtpm

and adding:

/run/libvirt/qemu/swtpm/*.pid rwk,

did the trick. Now the VM is running just fine.

TheJags commented 1 year ago

@lvoytek @stefanberger

Lastly, when I upgrade the system to say, Ubuntu MATE Lunar 23.04, or when the bug gets fixed...

should I keep this file (/etc/apparmor.d/local/usr.bin.swtpm), or it doesn't matter? Thanks.

lvoytek commented 1 year ago

@stefanberger

swtpm writes its own pid file but cannot do it due to ... what? Ownership of the directory? Having to overwrite an old pid file? What has changed since Jammy?

The swtpm repository is the same between Jammy and Kinetic. libvirt has some major changes though and it seems that within its handling of swtpm it now chowns the pid file to root permissions and leaves it behind after the vm shuts down.

In Jammy (libvirt version 8.0.0), sudo ls -la /run/libvirt/qemu/swtpm will show something like:

drwxrwx--- 2 libvirt-qemu swtpm  80 Nov 17 12:54 .
drwxr-xr-x 5 root         root  180 Nov 17 12:54 ..
-rw-r--r-- 1 swtpm        swtpm   5 Nov 17 12:54 14-win11-swtpm.pid
srw------- 1 libvirt-qemu kvm     0 Nov 17 12:54 14-win11-swtpm.sock

While in Kinetic (libvirt version 8.6.0) it shows something like:

drwxrwx--- 2 libvirt-qemu swtpm  80 Nov 17 13:34 .
drwxr-xr-x 5 root         root  180 Nov 17 13:34 ..
-rw-r--r-- 1 root         root    5 Nov 17 12:57 1-win11-swtpm.pid
-rw-r--r-- 1 root         root    5 Nov 17 13:34 2-win11-swtpm.pid
srw------- 1 libvirt-qemu kvm     0 Nov 17 13:34 2-win11-swtpm.sock

where previous pid files are not removed

@TheJags You can leave it as is and nothing will break when the fix arrives in Kinetic. I'll make a note in this bug report once its actually fixed though and you can remove it then if you'd like.

TheJags commented 1 year ago

@lvoytek

Thanks alot. I will keep the file as it is. You can close the issue.

andibing commented 1 year ago

Had exactly the same issue on Ubuntu 22.10 (albeit with kernel "6.1.0-060100rc5-generic") Only just encountered this thread after experimenting with many other options.

The AppArmor fix resolved the issue too. :-)

Thanks

@lvoytek @stefanberger

Thank you so much.

Creating (the file was not there)

sudo nano /etc/apparmor.d/local/usr.bin.swtpm

and adding:

/run/libvirt/qemu/swtpm/*.pid rwk,

did the trick. Now the VM is running just fine.

WXZhao7 commented 1 year ago

Met the same issue today after update libvirtd to 8.6.0. I just removed the TPM in the win11 client :- Now the VM worked with a new PIN. Host: Ubuntu20.04, Client: Win11-22H2

lvoytek commented 1 year ago

Hi @TheJags, just wanted to let you know the fix for this issue has been released in Ubuntu 22.04 and 22.10. Thanks!

stefanberger commented 1 year ago

@TheJags If this issue is resolved, can you close it?

TheJags commented 1 year ago

@stefanberger @lvoytek

Yes, thank you so much.