Closed nicolaspernoud closed 1 month ago
jimmykarily
commented
1 month ago The installation video shows an "enrolling measurements error" which might be the reason it doesn't boot after installation:
I don't know where it comes from yet but you can try to "Reset Secure Boot Keys" in qemu (and use "Standard Mode" in "Secure Boot Mode") :
you can then let the ISO auto enroll the keys (no need to enroll manually) just in case it has to do with the enrollment somehow.
nicolaspernoud
commented
1 month ago Unfortunately, that does not change anything. See the new video here :
https://github.com/user-attachments/assets/cc7555c0-9670-48c9-9051-2b67f58e837f
Itxaka
commented
1 month ago Problem seems to be the tpm2 libraries are missing on the debian image:
× systemd-pcrphase.service - TPM2 PCR Barrier (User)
Loaded: loaded (/lib/systemd/system/systemd-pcrphase.service; static)
Active: failed (Result: exit-code) since Mon 2024-07-29 08:40:28 UTC; 8min ago
Docs: man:systemd-pcrphase.service(8)
Main PID: 1506 (code=exited, status=1/FAILURE)
CPU: 15ms
Jul 29 08:40:28 localhost systemd[1]: Starting systemd-pcrphase.service - TPM2 PCR Barrier (User)...
Jul 29 08:40:28 localhost systemd-pcrphase[1506]: Failed to load TPM2 libraries: Operation not supported
Jul 29 08:40:28 localhost systemd[1]: systemd-pcrphase.service: Main process exited, code=exited, status=1/FAILURE
Jul 29 08:40:28 localhost systemd[1]: systemd-pcrphase.service: Failed with result 'exit-code'.
Jul 29 08:40:28 localhost systemd[1]: Failed to start systemd-pcrphase.service - TPM2 PCR Barrier (User).Also notice that Debian is not covered under the supported distros for UKI. we currently support and test Fedora and Ubuntu. Hopefully a small change to the Debian Dockerfile should make it work for UKI.
Itxaka
commented
1 month ago You can workaround this by adding the following line to your Dockerfile:
RUN apt-get update && apt-get install -y tpm2-* && apt-get clean && rm -rf /var/lib/apt/lists/*But that would still fail becuase Debian doesnt have the needed systemd-cryptenroll version
DBG debug from cryptenroll output="systemd-cryptenroll: unrecognized option '--tpm2-device-key=/run/systemd/tpm2-srk-public-key.tpm2b_public'\n"
root@localhost:~# systemd-cryptenroll --version
systemd 252 (252.26-1~deb12u2)
+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unifiedwe need at least systemd 255 for UKI to work correctly :)
nicolaspernoud
commented
1 month ago Thanks, I will try to use ubuntu as base image, it seems to be the simplest way. I will test it and close the issue if it works.
Itxaka
commented
1 month ago Had a quick look to see if this was feasible but there seems to be an issue with systemd-cryptenroll on testing.
Changes:
Dockerfile add tpm2 libs and crypsetup:
RUN apt-get update && apt-get install -y tpm2-* systemd-cryptsetup && apt-get clean && rm -rf /var/lib/apt/lists/*Fails to enroll measurements:
root@localhost:~# systemd-cryptenroll --tpm2-public-key-pcrs=11 --tpm2-pcrs= --t
pm2-signature=/run/systemd/tpm2-pcr-signature.json --tpm2-device-key=/run/system
d/tpm2-srk-public-key.tpm2b_public /dev/vda2
🔐 Please enter current passphrase for disk /dev/vda2: ••••
Assertion 'c' failed at src/shared/tpm2-util.c:2806, function tpm2_get_best_pcr_
bank(). Aborting.
Abortedversion
root@localhost:~# systemd-cryptenroll --version
systemd 256 (256.4-2)
+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +AC
L +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBCRYPTS
ETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ
+ZLIB +ZSTD +BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT +LIBARCHIVEopened an issue on systemd upstream to see if it comes from there as it looks like the pcrs are not being identified correctly: https://github.com/systemd/systemd/issues/33855
nicolaspernoud
commented
1 month ago I confirm that works with ubuntu. For further reference, here is the fully self sufficient script that create and start an ubuntu based kairos with qemu and emulated TPM :
#!/bin/bash
set -Eeuxo pipefail
WD="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
cd ${WD}
ISO_NAME="myiso"
mkdir -p ./files-iso
# Step 1: create the Dockerfile
cat <<EOF >./Dockerfile
FROM quay.io/kairos/ubuntu:24.04-core-amd64-generic-master
RUN echo "test" > /etc/test.txt
EOF
# Step 2: create the cloud init file
cat <<EOF >./files-iso/cloud_init.yaml
#cloud-config
install:
reboot: true
poweroff: false
auto: true # Required, for automated installations
bind_mounts:
- /var/lib/mycompany
users:
- name: kairos
passwd: kairos
write_files:
- path: /var/log/mycompany.log
content: |
# Mycompany cloud init done
EOF
# Step 3: generate the keys
sudo rm -rf ./keys
mkdir -p ./keys
MY_ORG="myorg"
docker run -v $PWD/keys:/work/keys -ti --rm quay.io/kairos/osbuilder-tools:latest genkey "$MY_ORG" --skip-microsoft-certs-I-KNOW-WHAT-IM-DOING --expiration-in-days 365 -o /work/keys
# Step 4: Build installable medium with keys
if [ ! -e "./build/${ISO_NAME}.iso" ]; then
IMAGE=${ISO_NAME}:latest
docker build --tag $IMAGE .
docker run \
-ti \
--rm \
-v /var/run/docker.sock:/var/run/docker.sock \
-v $PWD/build:/result \
-v $PWD/keys/:/keys \
-v $PWD/files-iso:/files-iso \
quay.io/kairos/osbuilder-tools:v0.300.3 \
build-uki $IMAGE \
--name "${ISO_NAME}" \
--overlay-iso /files-iso \
--boot-branding "My Custom Kairos" \
-t iso \
-d /result/ \
-k /keys
sudo chown -Rf $USER:$USER ./build
sudo chmod -Rf 777 ./build
fi
# Step 5: QEMU Test
MACHINE_NAME="test"
QEMU_IMG="${MACHINE_NAME}.img"
SSH_PORT="5555"
OVMF_CODE="/usr/share/OVMF/OVMF_CODE_4M.ms.fd"
OVMF_VARS_ORIG="/usr/share/OVMF/OVMF_VARS_4M.ms.fd"
OVMF_VARS="$(basename "${OVMF_VARS_ORIG}")"
if [ ! -e "${QEMU_IMG}" ]; then
qemu-img create -f qcow2 "${QEMU_IMG}" 40G
fi
if [ ! -e "${OVMF_VARS}" ]; then
cp "${OVMF_VARS_ORIG}" "${OVMF_VARS}"
fi
# TPM Emulator
mkdir -p /tmp/mytpm1
swtpm socket --tpmstate dir=/tmp/mytpm1 \
--ctrl type=unixio,path=/tmp/mytpm1/swtpm-sock \
--tpm2 \
--log level=20 &
# Start VM
qemu-system-x86_64 \
-enable-kvm \
-cpu host -smp cores=4,threads=1 -m 4096 \
-object rng-random,filename=/dev/urandom,id=rng0 \
-device virtio-rng-pci,rng=rng0 \
-name "${MACHINE_NAME}" \
-drive file="${QEMU_IMG}",format=qcow2 \
-net nic,model=virtio -net user,hostfwd=tcp::${SSH_PORT}-:22 \
-vga virtio \
-machine q35,smm=on \
-global driver=cfi.pflash01,property=secure,value=on \
-drive if=pflash,format=raw,unit=0,file="${OVMF_CODE}",readonly=on \
-drive if=pflash,format=raw,unit=1,file="${OVMF_VARS}" \
-chardev socket,id=chrtpm,path=/tmp/mytpm1/swtpm-sock \
-tpmdev emulator,id=tpm0,chardev=chrtpm \
-device tpm-tis,tpmdev=tpm0 \
-cdrom ./build/${ISO_NAME}.iso -boot menu=on -monitor stdio@Itxaka thanks a lot for your insights !
Itxaka
commented
1 month ago I changed the labels on it so we can refer to it later in case someone wants to work on adding Debian support to UKI
Itxaka
commented
1 month ago I confirm that works with ubuntu. For further reference, here is the fully self sufficient script that create and start an ubuntu based kairos with qemu and emulated TPM :
!/bin/bash
set -Eeuxo pipefail WD="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" cd ${WD}
ISO_NAME="myiso"
mkdir -p ./files-iso
Step 1: create the Dockerfile
cat <
./Dockerfile FROM quay.io/kairos/ubuntu:24.04-core-amd64-generic-master RUN echo "test" > /etc/test.txt EOF Step 2: create the cloud init file
cat <
./files-iso/cloud_init.yaml cloud-config
install: reboot: true poweroff: false auto: true # Required, for automated installations bind_mounts:
- /var/lib/mycompany
users:
- name: kairos passwd: kairos
write_files:
- path: /var/log/mycompany.log content: |
Mycompany cloud init done
EOF
Step 3: generate the keys
sudo rm -rf ./keys mkdir -p ./keys MY_ORG="myorg" docker run -v $PWD/keys:/work/keys -ti --rm quay.io/kairos/osbuilder-tools:latest genkey "$MY_ORG" --skip-microsoft-certs-I-KNOW-WHAT-IM-DOING --expiration-in-days 365 -o /work/keys
Step 4: Build installable medium with keys
if [ ! -e "./build/${ISO_NAME}.iso" ]; then IMAGE=${ISO_NAME}:latest docker build --tag $IMAGE . docker run \ -ti \ --rm \ -v /var/run/docker.sock:/var/run/docker.sock \ -v $PWD/build:/result \ -v $PWD/keys/:/keys \ -v $PWD/files-iso:/files-iso \ quay.io/kairos/osbuilder-tools:v0.300.3 \ build-uki $IMAGE \ --name "${ISO_NAME}" \ --overlay-iso /files-iso \ --boot-branding "My Custom Kairos" \ -t iso \ -d /result/ \ -k /keys sudo chown -Rf $USER:$USER ./build sudo chmod -Rf 777 ./build fi
Step 5: QEMU Test
MACHINE_NAME="test" QEMU_IMG="${MACHINE_NAME}.img" SSH_PORT="5555" OVMF_CODE="/usr/share/OVMF/OVMF_CODE_4M.ms.fd" OVMF_VARS_ORIG="/usr/share/OVMF/OVMF_VARS_4M.ms.fd" OVMF_VARS="$(basename "${OVMF_VARS_ORIG}")"
if [ ! -e "${QEMU_IMG}" ]; then qemu-img create -f qcow2 "${QEMU_IMG}" 40G fi
if [ ! -e "${OVMF_VARS}" ]; then cp "${OVMF_VARS_ORIG}" "${OVMF_VARS}" fi
TPM Emulator
mkdir -p /tmp/mytpm1 swtpm socket --tpmstate dir=/tmp/mytpm1 \ --ctrl type=unixio,path=/tmp/mytpm1/swtpm-sock \ --tpm2 \ --log level=20 &
Start VM
qemu-system-x86_64 \ -enable-kvm \ -cpu host -smp cores=4,threads=1 -m 4096 \ -object rng-random,filename=/dev/urandom,id=rng0 \ -device virtio-rng-pci,rng=rng0 \ -name "${MACHINE_NAME}" \ -drive file="${QEMU_IMG}",format=qcow2 \ -net nic,model=virtio -net user,hostfwd=tcp::${SSH_PORT}-:22 \ -vga virtio \ -machine q35,smm=on \ -global driver=cfi.pflash01,property=secure,value=on \ -drive if=pflash,format=raw,unit=0,file="${OVMF_CODE}",readonly=on \ -drive if=pflash,format=raw,unit=1,file="${OVMF_VARS}" \ -chardev socket,id=chrtpm,path=/tmp/mytpm1/swtpm-sock \ -tpmdev emulator,id=tpm0,chardev=chrtpm \ -device tpm-tis,tpmdev=tpm0 \ -cdrom ./build/${ISO_NAME}.iso -boot menu=on -monitor stdio
@Itxaka thanks a lot for your insights !
Very handy, will come into use for https://github.com/kairos-io/kairos/issues/2173 for sure. Thanks @nicolaspernoud !
Kairos version: FROM quay.io/kairos/debian:bookworm-core-amd64-generic-master quay.io/kairos/osbuilder-tools:v0.300.3
CPU architecture, OS, and Version: 6.5.0-45-generic #45~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Mon Jul 15 16:40:02 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
Describe the bug When trying to create a custom image using secure boot with the script below, enrolling the db in the bios, and starting kairos, it get stuck after reboot.
To Reproduce Use the script below, it should be self sufficient :
Expected behavior OS starting
Logs See video of boot. https://github.com/user-attachments/assets/a05426cc-9ef1-4b93-a29b-804b877b5cba