search

RPi-Distro / repo

Issue tracking for the archive.raspberrypi.com repo
38 stars 1 forks source link

[Bookworm] cannot unlock LUKS2 volume from initramfs #362

Open bklop opened 10 months ago

bklop commented 10 months ago

With RBPI OS Lite 64 based on Debian 11 (2023-05-03-raspios-bullseye-arm64-lite.img.xz) running on a RBPI CM4 I can decrypt a LUKS2 volume encrypted with Adiantum from initramfs. The procedure I follow for building initramfs:

sudo apt install busybox cryptsetup initramfs-tools

Check that the algorithms are available: cryptsetup benchmark -c xchacha20,aes-adiantum-plain64

/etc/initramfs-tools/hooks/luks_hooks

#!/bin/sh -e
PREREQS=""
case $1 in
        prereqs) echo "${PREREQS}"; exit 0;;
esac

. /usr/share/initramfs-tools/hook-functions

copy_exec /sbin/resize2fs /sbin
copy_exec /sbin/fdisk /sbin
copy_exec /sbin/cryptsetup /sbin

sudo chmod +x /etc/initramfs-tools/hooks/luks_hooks

/etc/initramfs-tools/modules

algif_skcipher
xchacha20
adiantum
aes_arm
sha256
nhpoly1305
dm-crypt

Finally, build a new initramfs: sudo -E CRYPTSETUP=y mkinitramfs -o /boot/initramfs.gz

I then update /boot/cmdline.txt etc. And during reboot I get bumped into a shell where I can unlock a volume: cryptsetup luksOpen /dev/XYZ crypted

This works fine with Debian 11. When I followed the same procedure with Bookworm (2023-12-11-raspios-bookworm-arm64-lite.img.xz) everything works fine until I boot into initramfs and try to unlock the volume, where I got this error:

_libgcc_s.so.1 must be installed for pthreadexit to work

I realized it was most likely this issue: https://bugs.launchpad.net/ubuntu/+source/cryptsetup/+bug/1960080

So I added libpthread and libgcc to my luks_hooks script:

#!/bin/sh -e
PREREQS=""
case $1 in
        prereqs) echo "${PREREQS}"; exit 0;;
esac

. /usr/share/initramfs-tools/hook-functions

copy_exec /usr/lib/aarch64-linux-gnu/libgcc_s.so.1 /usr/lib/aarch64-linux-gnu
copy_exec /usr/lib/aarch64-linux-gnu/libpthread.so.0 /usr/lib/aarch64-linux-gnu
copy_exec /sbin/resize2fs /sbin
copy_exec /sbin/fdisk /sbin
copy_exec /sbin/cryptsetup /sbin

After building a new initramfs I then checked the archive for the presence of the modules:

lsinitramfs /boot/initramfs.gz | grep -P "libgcc"
lsinitramfs /boot/initramfs.gz | grep -P "sbin/(cryptsetup|resize2fs|fdisk)"
lsinitramfs /boot/initramfs.gz | grep -P "(algif_skcipher|chacha|adiantum|aes-arm|sha256|nhpoly1305|dm-crypt)"

However, after booting into initramfs I still cannot use cryptfs to unlock the volume: Cannot use xchacha20,aes-adiantum-plain64 cipher

XECDesign commented 10 months ago

@tdewey-rpi, since you're looking at something similar, is this something you've stumbled upon?

tdewey-rpi commented 10 months ago

ACK.

I'm going to try and reproduce something along these lines now.

tdewey-rpi commented 10 months ago

Thanks for the report, @blkop.

I've been able to reproduce part of your failure and methodology. I can confirm I'm seeing files explicitly named in the hooks file be transferred to the resultant initramfs, but it appears that the documentation for mkinitramfs isn't aligning with observed behaviour on bookworm. Specifically, it says:

mkinitramfs will automatically detect which libraries the executable depends on and copy them to the initramfs

And sure enough, the adiantum module isn't being copied into the initramfs, while it's present on the installed system:

initramfs: usr/lib/modules/6.1.0-rpi8-rpi-2712/kernel/arch/arm64/crypto/chacha-neon.ko usr/lib/modules/6.1.0-rpi8-rpi-2712/kernel/arch/arm64/crypto/poly1305-neon.ko usr/lib/modules/6.1.0-rpi8-rpi-2712/kernel/arch/arm64/lib/xor-neon.ko usr/lib/modules/6.1.0-rpi8-rpi-2712/kernel/crypto/blake2b_generic.ko usr/lib/modules/6.1.0-rpi8-rpi-2712/kernel/crypto/xor.ko usr/lib/modules/6.1.0-rpi8-rpi-2712/kernel/lib/crypto/libarc4.ko usr/lib/modules/6.1.0-rpi8-rpi-2712/kernel/lib/crypto/libchacha.ko usr/lib/modules/6.1.0-rpi8-rpi-2712/kernel/lib/crypto/libchacha20poly1305.ko usr/lib/modules/6.1.0-rpi8-rpi-2712/kernel/lib/crypto/libcurve25519-generic.ko

Installed system: tdewey@tdewey-pi:/usr/lib/modules/6.1.0-rpi8-rpi-2712/kernel/arch/arm64/crypto $ ls aes-arm64.ko.xz aes-ce-ccm.ko.xz aes-neon-blk.ko.xz chacha-neon.ko.xz poly1305-neon.ko.xz sha256-arm64.ko.xz sha3-ce.ko.xz sha512-ce.ko.xz sm4-ce-cipher.ko.xz aes-ce-blk.ko.xz aes-ce-cipher.ko.xz aes-neon-bs.ko.xz ghash-ce.ko.xz sha1-ce.ko.xz sha2-ce.ko.xz sha512-arm64.ko.xz sm3-ce.ko.xz tdewey@tdewey-pi:/usr/lib/modules/6.1.0-rpi8-rpi-2712/kernel/arch/arm64/crypto $ ls /usr/lib/modules/6.1.0-rpi8-rpi-2712/kernel/crypto adiantum.ko.xz algif_hash.ko.xz authencesn.ko.xz cast_common.ko.xz chacha_generic.ko.xz ctr.ko.xz echainiv.ko.xz ghash-generic.ko.xz md4.ko.xz poly1305_generic.ko.xz sm4.ko.xz xcbc.ko.xz aes_generic.ko.xz algif_rng.ko.xz authenc.ko.xz cbc.ko.xz cmac.ko.xz drbg.ko.xz essiv.ko.xz hmac.ko.xz md5.ko.xz seqiv.ko.xz twofish_common.ko.xz xor.ko.xz af_alg.ko.xz algif_skcipher.ko.xz blake2b_generic.ko.xz ccm.ko.xz cryptd.ko.xz ecc.ko.xz gcm.ko.xz jitterentropy_rng.ko.xz michael_mic.ko.xz sha3_generic.ko.xz twofish_generic.ko.xz xxhash_generic.ko.xz algif_aead.ko.xz async_tx cast5_generic.ko.xz chacha20poly1305.ko.xz crypto_user.ko.xz ecdh_generic.ko.xz gf128mul.ko.xz lz4.ko.xz nhpoly1305.ko.xz sm3.ko.xz wp512.ko.xz zstd.ko.xz tdewey@tdewey-pi:/usr/lib/modules/6.1.0-rpi8-rpi-2712/kernel/arch/arm64/crypto $ ls /usr/lib/modules/6.1.0-rpi8-rpi-2712/kernel/lib/crypto libaes.ko.xz libarc4.ko.xz libchacha20poly1305.ko.xz libchacha.ko.xz libcurve25519-generic.ko.xz libcurve25519.ko.xz libpoly1305.ko.xz

Have you tried adding the kernel modules to /etc/initramfs-tools/modules? Having done so on my build, I now see those objects where I'd expect.

Just saw your modules file - the only way I can investigate further is to spin up another device, so I'll schedule this digging.

tdewey-rpi commented 10 months ago

Can you confirm you have libpoly1305.ko in your initramfs?

bklop commented 9 months ago

Yes libpoly1305.ko was copied:

pi@raspberrypi:~ $ lsinitramfs /boot/initramfs.gz | grep -P "(cryptsetup|adiantum|libpoly1305)"
usr/lib/aarch64-linux-gnu/libcryptsetup.so.12
usr/lib/aarch64-linux-gnu/libcryptsetup.so.12.9.0
usr/lib/modules/6.1.0-rpi7-rpi-v8/kernel/crypto/adiantum.ko
usr/lib/modules/6.1.0-rpi7-rpi-v8/kernel/lib/crypto/libpoly1305.ko
bklop commented 7 months ago

@tdewey-rpi did you get a chance to look into this?

AutumnSpark1226 commented 6 months ago

I followed the same (I guess; this one? https://rr-developer.github.io/LUKS-on-Raspberry-Pi/) tutorial and got similar errors. Firstly, I noticed that everthing that was in /boot in the tutorial must now be in /boot/firmware. You can move /boot/initramfs.gz to /boot/firmware/initramfs.gz Secondly, on 64bit RasPi OS installations the aes_arm module doesn't exist. Instead you should use the aes_arm64 module. Manually adding libgcc_s.so.1 and libpthread.so.0 is still required.

marsante commented 3 months ago

@bklop According to /etc/initramfs-tools/modules https://alioth-lists.debian.net/pipermail/pkg-cryptsetup-devel/2023-January/009588.html you can add

aes_generic
chacha_generic
nhpoly1350
xts

in /etc/initramfs-tools/modules to fix:

However, after booting into initramfs, I still cannot use cryptfs to unlock the volume: Cannot use xchacha20, aes-adiantum-plain64 encryption

Despite all the fixes I've made, when i unlocked luks on initramfs and typed exit initramfs opens with ALERT! root=/dev/mapper/sdcard does not exist. Switching to a shell. ls /dev/mapper/sdcard shows it is present oddly.

exander77 commented 2 months ago

I am having exactly the same issue.

I don't understand how can it write that /dev/mapper/XXX is not present if I see it is there. What kind of nonsensical error message is that?

exander77 commented 2 months ago

Seems culprit is this function:

# Resolve device node from a name.  This expands any LABEL or UUID.
# $1=name
# Resolved name is echoed.
resolve_device() {
        DEV="$1"

        case "$DEV" in
        LABEL=* | UUID=* | PARTLABEL=* | PARTUUID=*)
                DEV="$(blkid -l -t "$DEV" -o device)" || return 1
                ;;
        esac
        [ -e "$DEV" ] && echo "$DEV"
}

It doesn't handle case where device is already resolved, like:

root=/dev/mapper/XXX
exander77 commented 2 months ago

Fix:

resolve_device() {
    DEV="$1"

    case "$DEV" in
    /dev/*)
        # If the input starts with /dev/, we don't need to do anything else
        ;;
    LABEL=* | UUID=* | PARTLABEL=* | PARTUUID=*)
        DEV="$(blkid -l -t "$DEV" -o device)" || return 1
        ;;
    esac

    [ -e "$DEV" ] && echo "$DEV"
}
marsante commented 2 months ago

Thanks for sharing a fix. It's located in /usr/share/initramfs-tools/scripts/functions ? I’ll be able to test it in a week.

exander77 commented 2 months ago

Thanks for sharing a fix. It's located in /usr/share/initramfs-tools/scripts/functions ? I’ll be able to test it in a week.

Yes. For practical reasons I do this:

sudo sed -i '/^resolve_device()/,/^}/c\resolve_device() {\n    DEV="\$1"\n\n    case "\$DEV" in\n    /dev/*)\n        ;;\n    LABEL=* | UUID=* | PARTLABEL=* | PARTUUID=*)\n        DEV="\$(blkid -l -t "\$DEV" -o device)" || return 1\n        ;;\n    esac\n\n    [ -e "\$DEV" ] && echo "\$DEV"\n}' /mnt/root/usr/share/initramfs-tools/scripts/functions

Replacing entire function, /mnt/root is the location where I have root file system mounted.

exander77 commented 2 months ago

@marsante Btw, what are you doing? I am currently making script that creates mdadm raid1, and luks on top of that and makes it a raspberry pi root. I have 2x m.2 on Raspberry Pi. seems to work fine after I fixed this.

marsante commented 2 months ago

@exander77 much simpler than you, just encrypting the SD card.

marsante commented 2 months ago

I can test, it works, but every time I reboot, I have to reopen luks via initramfs, maybe I missed something ?

Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /script/local-premount ... done.
Begin: Waiting for root file system ... Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
...
exander77 commented 2 months ago

I can test, it works, but every time I reboot, I have to reopen luks via initramfs, maybe I missed something ?

Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /script/local-premount ... done.
Begin: Waiting for root file system ... Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
...

I am not sure what you mean by this.

marsante commented 2 months ago

Sorry if it's not clear . I apply your fix

Fix:

resolve_device() {
    DEV="$1"

    case "$DEV" in
    /dev/*)
        # If the input starts with /dev/, we don't need to do anything else
        ;;
    LABEL=* | UUID=* | PARTLABEL=* | PARTUUID=*)
        DEV="$(blkid -l -t "$DEV" -o device)" || return 1
        ;;
    esac

    [ -e "$DEV" ] && echo "$DEV"
}

After that I can finish all the procedures in initramfs, exit it and get into the OS. But if I reboot, the above error message appears and I get into initramfs. I can run

cryptsetup luksOpen /dev/mmcblk0p2 sdcard
exit

to boot into the OS.

exander77 commented 2 months ago 
cryptsetup luksOpen /dev/mmcblk0p2 sdcard
exit

to boot into the OS.

That would be expected if nothing else is calling luksOpen. Do you have some script in initramfs to open luks? If you don't want to write it yourself, you can install cryptsetup-initramfs, that installs such script into initramfs.

exander77 commented 2 months ago

If you list the contents, you can see what it does:

$ apt-file list cryptsetup-initramfs
cryptsetup-initramfs: /etc/cryptsetup-initramfs/conf-hook
cryptsetup-initramfs: /usr/share/bug/cryptsetup-initramfs
cryptsetup-initramfs: /usr/share/cryptsetup/initramfs/bin/cryptroot-unlock
cryptsetup-initramfs: /usr/share/doc/cryptsetup-initramfs/README.initramfs.gz
cryptsetup-initramfs: /usr/share/doc/cryptsetup-initramfs/changelog.Debian.gz
cryptsetup-initramfs: /usr/share/doc/cryptsetup-initramfs/copyright
cryptsetup-initramfs: /usr/share/initramfs-tools/conf-hooks.d/cryptsetup
cryptsetup-initramfs: /usr/share/initramfs-tools/hooks/cryptgnupg
cryptsetup-initramfs: /usr/share/initramfs-tools/hooks/cryptgnupg-sc
cryptsetup-initramfs: /usr/share/initramfs-tools/hooks/cryptkeyctl
cryptsetup-initramfs: /usr/share/initramfs-tools/hooks/cryptopensc
cryptsetup-initramfs: /usr/share/initramfs-tools/hooks/cryptpassdev
cryptsetup-initramfs: /usr/share/initramfs-tools/hooks/cryptroot
cryptsetup-initramfs: /usr/share/initramfs-tools/hooks/cryptroot-unlock
cryptsetup-initramfs: /usr/share/initramfs-tools/scripts/local-block/cryptroot
cryptsetup-initramfs: /usr/share/initramfs-tools/scripts/local-bottom/cryptgnupg-sc
cryptsetup-initramfs: /usr/share/initramfs-tools/scripts/local-bottom/cryptopensc
cryptsetup-initramfs: /usr/share/initramfs-tools/scripts/local-bottom/cryptroot
cryptsetup-initramfs: /usr/share/initramfs-tools/scripts/local-top/cryptopensc
cryptsetup-initramfs: /usr/share/initramfs-tools/scripts/local-top/cryptroot
cryptsetup-initramfs: /usr/share/lintian/overrides/cryptsetup-initramfs

Basically, it installs a hook and unlock script that calls luksOpen.

marsante commented 2 months ago

Oh thanks, I thought it was a recommended package with cryptsetup, but I have to install it manually. I'll be able to search in the installed files, because it unfortunately doesn't change the behavior at boot.

marsante commented 2 months ago

Ok I think I got it. First time I made.

sudo update-initramfs -u
sudo -E CRYPTSETUP=y mkinitramfs -o /boot/firmware/initramfs.gz

This time I make : 

sudo update-initramfs -c -k $(uname -r)
sudo -E CRYPTSETUP=y mkinitramfs -o /boot/firmware/initramfs.gz

I don't know if sudo -E CRYPTSETUP=y mkinitramfs -o /boot/firmware/initramfs.gz command was necessary in addition to sudo update-initramfs -c -k $(uname -r) but at the end it works.

Thank again for your help @exander77

exander77 commented 2 months ago

Ok I think I got it. First time I made.

sudo update-initramfs -u
sudo -E CRYPTSETUP=y mkinitramfs -o /boot/firmware/initramfs.gz

This time I make :

sudo update-initramfs -c -k $(uname -r)
sudo -E CRYPTSETUP=y mkinitramfs -o /boot/firmware/initramfs.gz

I don't know if sudo -E CRYPTSETUP=y mkinitramfs -o /boot/firmware/initramfs.gz command was necessary in addition to sudo update-initramfs -c -k $(uname -r) but at the end it works.

Thank again for your help @exander77

Btw, I don't think you have to set up CRYPTSETUP=y on command line. You may be missing it in configuration if you actually need it. You definitely need to rebuild initramfs with update-initramfs tools.