Requesting fix for Cryptsetup disk encryption build feature (seems broken)

[xmijo]

The cryptroot build feature seems broken, or possibly only the SSH-unlock part of it. I am unable to verify if it works without SSH-unlock enabled due to a headless setup.

Based on this issue described on the Armbian forum:

I'm attempting to build an image for the ROC-RK3328-CC (Renegade) with cryptroot encryption (and SSH unlock) enabled, but I run into weird behavior.

The board will start and I can SSH in and reach the BusyBox prompt. So far so good but from what I've gathered I should enter the command "unlock" there to display the cryptroot password prompt. However it only results in the message -sh: unlock: not found. I read somewhere else the command might be "cryptroot-unlock" but that instead gives nothing, not even the error message.

Furthermore, regardless whether I enter any commands, around 10 seconds into the SSH session it will freeze and I eventually get kicked off with a broken pipe error. I suspect the board is restarting by itself but it is difficult to verify because it is running headless. After a few seconds I'm able to SSH in again for another ~10 sec session.

Building it without cryptroot works just fine and the board starts normally and I can SSH in. I have not tested building it with cryptroot only (without the SSH-unlock ability), because again it is difficult to verify due to the headless setup.

Has anyone run into these kind of behavior when building Armbian images with cryptroot setup or have any idea what's gone wrong? Is it a bug in the build cryptroot feature?

Below is my config-default.conf

KERNEL_ONLY="no" KERNEL_CONFIGURE="no" CLEAN_LEVEL="make,debs,oldcache" DEST_LANG="en_US.UTF-8" EXTERNAL_NEW="prebuilt" INSTALL_HEADERS="yes" LIB_TAG="master" USE_TORRENT="yes" CARD_DEVICE="/dev/sdb" BOARD="renegade" RELEASE="buster" BUILD_MINIMAL="yes" CRYPTROOT_ENABLE="yes" CRYPTROOT_PASSPHRASE="password" CRYPTROOT_SSH_UNLOCK="yes" CRYPTROOT_SSH_UNLOCK_PORT="2222"

Can anyone confirm there's an issue by replicating? Basically cryptsetup and ssh-unlock enabled.

Pinging authors of related merged pulls to see if they (or anyone else with the know-how) has any idea for a fix. @zciendor #947 #948 #1069 @briaeros #1017 @ei-ke #1182

[ei-ke]

@xmijo Sorry to hear that. I've only added the CRYPTROOT_PARAMETERS for my personal needs. I'm using Armbian on Odroids and don't have a ROC-RK3328-CC. I'll set up a build environment tomorrow and see if the build with your parameters will work on an Odroid.

[xmijo]

Thanks, I greatly appreciate your help. I regret that I'm unable to diagnose it further because of the headless setup, lacking a screen at the moment but might be able to get access to one in a few weeks for further investigation.

[ei-ke]

@xmijo No problem. I can only test if it works or not. Are you connecting to dropbear by means of public key authentication by placing it in userpatches/dropbear_authorized_keys (which i do) or by using the one that is created when building the image? Just asking to reproduce your setup.

[xmijo]

I use the one created when building the image, located in output/images/

Then I SSH in with the root account

[BlankScreen2019]

@ei-ke Hi there,

I just tried to get an encrypted System up and running on an Odroid HC2. So I did what you planned to do.

The build-System was a fresh installed & updated Ubuntu 18.04.3 (no docker or stuff like that)

I've choosen the Armbian_5.98Odroidxu4Debian_stretch_default_4.14.141 because i suppose that is a stable & known system.

first build was just 'plain': -> Full OS
-> vendor-Kernel

[ o.k. ] Repeat Build Options [ ./compile.sh BOARD=odroidxu4 BRANCH=default RELEASE=stretch BUILD_MINIMAL=no BUILD_DESKTOP=no KERNEL_ONLY=no KERNEL_CONFIGURE=yes ]

This runs without any problems. (As far as i can tell from a single login , password change and user setup)

The second build was with the following options:

./compile.sh CRYPTROOT_ENABLE=yes CRYPTROOT_PASSPHRASE="MYSECRECTPASS" CRYPTROOT_SSH_UNLOCK=yes CRYPTROOT_SSH_PORT=2222

This build doesn't work. I cannot login via ssh:

sudo cp /home/user/build/output/images/Armbian_5.98_Odroidxu4_Debian_stretch_default_4.14.141.key /home/user/.ssh/v2.key

user@user-desktop:~/.ssh$ ssh -p2222 -vvvv -i /home/user/.ssh/v2.key root@10.10.10.99 OpenSSH_7.6p1 Ubuntu-4ubuntu0.3, OpenSSL 1.0.2n 7 Dec 2017 debug1: Reading configuration data /etc/ssh/ssh_config debug1: /etc/ssh/ssh_config line 19: Applying options for * debug2: resolving "10.10.10.99" port 2222 debug2: ssh_connect_direct: needpriv 0 debug1: Connecting to 10.10.10.99 [10.10.10.99] port 2222. debug1: connect to address 10.10.10.99 port 2222: Connection refused ssh: connect to host 10.10.10.99 port 2222: Connection refused

A ping works fine, so the Odroid is running with network connetion:

user@user-desktop:~$ ping 10.10.10.99 PING 10.10.10.99 (10.10.10.99) 56(84) bytes of data. 64 bytes from 10.10.10.99: icmp_seq=1 ttl=64 time=0.636 ms 64 bytes from 10.10.10.99: icmp_seq=2 ttl=64 time=0.605 ms 64 bytes from 10.10.10.99: icmp_seq=3 ttl=64 time=0.611 ms 64 bytes from 10.10.10.99: icmp_seq=4 ttl=64 time=0.615 ms

checking via nmap concludes that there is nothing listening at port 2222

user@user-desktop:~$ nmap -p 22,2222 -T4 -A -v 10.10.10.99 Starting Nmap 7.60 ( https://nmap.org ) at 2019-10-06 14:37 CEST [...] Scanning 10.10.10.99 [2 ports] [...] Host is up (0.00064s latency). [...] PORT STATE SERVICE VERSION 22/tcp closed ssh 2222/tcp closed EtherNetIP-1 [...] Nmap done: 1 IP address (1 host up) scanned in 0.33 seconds

Hope that helped a little bit.

[xmijo]

@BlankScreen2019 Thx for testing and reporting back.

In my case I am able to connect to the port (2222) and reach the BusyBox prompt.

[BlankScreen2019]

Hm. Im still struggeling with the ssh login. But from trying to set up an encrypted OMV (based on armbian) i remember that i got an error like this:

WARNING: Setting CRYPTSETUP in /etc/initramfs-tools/initramfs.conf is deprecated and will stop working in the future. Use /etc/cryptsetup-initramfs/conf-hook instead.

when installing the "full disk encryption" by hand.

Perhaps that is a lead to follow.

[BlankScreen2019]

I can confirm that building an Debain stretch image for a OdroidHC2 (OdroidXU4 group) withing vagrant/virtualbox on a fresh Ubuntu 18.04 system works!

./compile.sh CRYPTROOT_ENABLE=yes CRYPTROOT_PASSPHRASE="MYSECRECTPASS" CRYPTROOT_SSH_UNLOCK=yes CRYPTROOT_SSH_PORT=2222 2>&1 | tee vagrant_crypt_build.log

For some strange reason i had to ssh into it via port 2022 .

ssh root@192.168.15.123 -p 2022 -i Armbian_5.98_Odroidxu4_Debian_stretch_default_4.14.141.key

You quoted:

The board will start and I can SSH in and reach the BusyBox prompt. So far so good but from what I've gathered I should enter the command "unlock" there to display the cryptroot password prompt. However it only results in the message -sh: unlock: not found. I read somewhere else the command might be "cryptroot-unlock" but that instead gives nothing, not even the error message.

For me it was cryptroot-unlock do get the magic done.

[xmijo]

I will test port 2022 as well but I don't think it is related since I do reach the BusyBox prompt on port 2222, as I had specified during build. But it is strange that it opens port 2022 for you after you specified 2222, do you have any idea why this would happen?

I do notice that you are building it for Stretch while I was doing it for Buster. Perhaps the cryptroot part of the build script needs to be updated for compatibility with Buster? I should test building for Stretch and see if it works then.

[BlankScreen2019]

Hi There,

no i have no clue why it is on Port 2022... Port 2222 does not function at all. Perhaps it was just a typo i made..

I just build a Buster image for testing. I also got stuck in busybox ... When entering:

cryptroot-unlock

nothing happens and i am not asked for the password. So i can confirm that my image build for buster on Odroid HC2 does not work.

As far as i figure the crytroot-unlock is a wrapper for cryptsetup. It seems that something changed in using cryptsetup... Which would lead me to the already mentiond:

WARNING: Setting CRYPTSETUP in /etc/initramfs-tools/initramfs.conf is deprecated and will stop working in the future. Use /etc/cryptsetup-initramfs/conf-hook instead.

[xmijo]

Thanks for your testing.

I investigated further and it appears the Cryptsetup version in Buster has been bumped to v2.1 (Stretch uses v1.7) so I imagine the issue is related to this. The new Cryptsetup uses LUKS2 instead of LUKS1, among other changes.

I'm unfortunately unable to dive deeper into this at the moment (in the middle of relocating abroad), but if anyone else is able to work on a fix, a good place to start might be the info stated in the Buster release notes where they also link to instructions on how to install Debian 10 with encrypted boot.

[xmijo]

More clues-- The build install.log shows some errors.

Preparing to unpack .../linux-image-rockchip64_5.98_arm64.deb ... Unpacking linux-image-rockchip64 (5.98) ... Setting up linux-image-rockchip64 (5.98) ... qemu: Unsupported syscall: 276 update-initramfs: Generating /boot/initrd.img-4.4.192-rockchip64 cryptsetup: WARNING: Couldn't determine root device cryptsetup: WARNING: The initramfs image may not contain cryptsetup binaries nor crypto modules. If that's on purpose, you may want to uninstall the 'cryptsetup-initramfs' package in order to disable the cryptsetup initramfs integration and avoid this warning. W: Couldn't identify type of root file system for fsck hook qemu: Unsupported syscall: 276

Preparing to unpack .../linux-buster-root-renegade_5.98_arm64.deb ... Adding 'diversion of /etc/mpv/mpv.conf to /etc/mpv/mpv-dist.conf by linux-buster-root-renegade' Unpacking linux-buster-root-renegade (5.98) ... Setting up linux-buster-root-renegade (5.98) ... Created symlink /etc/systemd/system/sysinit.target.wants/armbian-ramlog.service -> /lib/systemd/system/armbian-ramlog.service. qemu: Unsupported syscall: 276 qemu: Unsupported syscall: 276 qemu: Unsupported syscall: 276 qemu: Unsupported syscall: 276 Processing triggers for initramfs-tools (0.133+deb10u1) ... update-initramfs: Generating /boot/initrd.img-4.4.192-rockchip64 cryptsetup: WARNING: Couldn't determine root device cryptsetup: WARNING: The initramfs image may not contain cryptsetup binaries nor crypto modules. If that's on purpose, you may want to uninstall the 'cryptsetup-initramfs' package in order to disable the cryptsetup initramfs integration and avoid this warning. W: Couldn't identify type of root file system for fsck hook qemu: Unsupported syscall: 276 update-initramfs: Converting to u-boot format Running in chroot, ignoring request: daemon-reload

Calling hook cryptroot cryptsetup: WARNING: Couldn't determine root device cryptsetup: WARNING: The initramfs image may not contain cryptsetup binaries nor crypto modules. If that's on purpose, you may want to uninstall the 'cryptsetup-initramfs' package in order to disable the cryptsetup initramfs integration and avoid this warning. Calling hook cryptroot-unlock Adding script /usr/share/cryptsetup/initramfs/bin/cryptroot-unlock

Calling hook klibc-utils /usr/share/initramfs-tools/scripts/init-top/ORDER ignored: not executable /usr/share/initramfs-tools/scripts/local-premount/ORDER ignored: not executable /usr/share/initramfs-tools/scripts/local-top/ORDER ignored: not executable /usr/share/initramfs-tools/scripts/init-bottom/ORDER ignored: not executable /usr/share/initramfs-tools/scripts/local-bottom/ORDER ignored: not executable /usr/share/initramfs-tools/scripts/local-block/ORDER ignored: not executable /usr/share/initramfs-tools/scripts/init-premount/ORDER ignored: not executable Building cpio /boot/initrd.img-4.4.192-rockchip64.new initramfs qemu: Unsupported syscall: 276 update-initramfs: Converting to u-boot format

[zciendor]

This seems to be the problem:

cryptsetup: ERROR: Couldn't resolve device tmpfs
cryptsetup: WARNING: Couldn't determine root device

I can only speculate. Could be that cryptsetup-initramfs in Debian buster is now using /proc/mounts instead of /etc/fstab to locate the root device. And now it has issues with the tmpfs that is used by Armbian at build time for the root device, which is different from the real root device at run time.

At build time when update-initramfs runs, /proc/mounts looks likte this:

tmpfs / tmpfs rw,relatime,size=10933248k 0 0
chproc /proc proc rw,relatime 0 0
chsys /sys sysfs rw,relatime 0 0
chdev /dev devtmpfs rw,relatime,size=8129932k,nr_inodes=2032483,mode=755 0 0
chpts /dev/pts devpts rw,relatime,mode=600,ptmxmode=000 0 0

There's no way cryptsetup-initramfs can match this with what is specified in /etc/crypttab using the device mapper (/dev/mapper/). Hence, we get the error above I guess...

One solution could be to mount the root partition of the final image ($SDCARD) using cryptsetup (instead of tmpfs) so the device mapper name matches how it will look at run time, before update-initramfs is called the last time in debbootstrap.sh.

[igorpecovnik]

One solution could be to mount the root partition of the final image ($SDCARD) using cryptsetup (instead of tmpfs) so the device mapper name matches how it will look at run time, before update-initramfs is called the last time in debbootstrap.sh.

Try. Even a dirty workaround is better then the current situation.