misc: write-up for install to emmc

[oswalpalash]

Hello,

I have an oak chromebook and I was trying to follow https://github.com/hexdump0815/imagebuilder/blob/main/info/generic/install-to-emmc-with-luks-full-disk-encryption.txt to use the usb booted system to partition and rsync to the emmc. I've tried a few combinations of partitions but none of them seem to boot. Is there a possibility of splitting this doc into 3 separate ones -

1. follow this to partition and install w/o efi
2. follow this to partition and install with efi
3. non chromebook writeup

Also, an observation - in the two different builds of oak (focal and bullseye), cgpt worked differently and a different set of sector count had to be used to achieve partitions. At this point, I would love any advice that can help me get going to set up the emmc (with or without luks+btrfs). I am ready to do a proper write-up if I can get this working for me. Any advice would be appreciated; and you've already done so much good work :) Thanks

[Geofferey]

I'm not sure if this will apply to you but I took a much simpler approach for a arm64 device (trogdor). Write the image to USB drive and store a copy of that same image you used to flash drive in changeme's home dir on USB drive. Boot the drive on your device and write the image to /dev/mmcblk1 (in my case) using dd. If you can't find a way to get image on drive before booting put it on another and plug that it in while booted from sd/USB to have img available for writing.

dd if=~/chromebook-linux.img of=/dev/mmcblk1

Writing the image this way should partition the drive and everything so you don't need to do so manually. Afterwards make sure to remove the USB before next boot and expand volume etc on first boot from emmc using /scripts/extend-rootfs.sh. Be sure to remove USB/SD before booting from internel or it may use the data located on your initial media source.

I can not guarantee this will work for you, nor can I guarantee it's appropriate for your device. It did work for me and my Chromebook seems to be running just fine. YMMV.

[oswalpalash]

That definitely worked @Geofferey ! Thank you so much :)

[Geofferey]

@oswalpalash No problem man. I was struggling too because I'm used to Android and a move like that would result in a very expensive paper weight 😂 . Have fun!

[hexdump0815]

@oswalpalash - i agree with you that the "documentation" had definitiely a lot of room for improvement, but i preferred to initially at least dump what i had in mind and to improve it later ... in theory it should work - i'm actually setting up all systems i'm installing fresh by copy/pasting from this guids, so at least bullseye is regularly tested - maybe i'll have to look at focal ... i plan to add ubuntu jammy soon anyway, so a good chance to test this while doing so ...

@Geofferey - thanks a lot for proposing an intermediate solution, which seems to help working around potential problems

best wishes - hexdump

[Geofferey]

@hexdump0815 I think I might fork things and start helping with documentation, would you be willing to merge if I do so and quality is good? We already have two things to add 😜

[hexdump0815]

@Geofferey - sure that is a good idea - thanks a lot for the offer - help is welcome

[leezu]

For reference, flashing the usb / sdcard and copying/storing a copy of that same image you used to flash the drive can be achieved via:

gunzip -c chromebook_trogdor-aarch64-bullseye.img.gz | sudo dd of=/dev/sdX status=progress
sudo growpart /dev/sdX 4
sudo mount /dev/sdX4 /mnt
sudo btrfs filesystem resize max /mnt
sudo cp chromebook_trogdor-aarch64-bullseye.img.gz /mnt

As btrfs is used, a separate btrfs filesystem resize is needed in addition to growpart avoid unused slack in the btrfs partition.

[leezu]

@hexdump0815 I was able to reproduce the issue faced by @oswalpalash, ie that https://github.com/hexdump0815/imagebuilder/blob/main/info/generic/install-to-emmc-with-luks-full-disk-encryption.txt style installation fails to boot.

I also tried updating the kernel to 5.18-rc5 based on kernel build notes, but the new kernel failed to boot (for which test-booting-a-kernel-on-chromebooks.txt was very helpful - thank you!).

However, clearly your image at https://github.com/hexdump0815/imagebuilder/releases/tag/211120-02 does work. Do you recall how the kernel packing step you performed differs from the kernel build notes:

# start chromebook special - required: apt-get install liblz4-tool vboot-kernel-utils
cp arch/arm64/boot/Image Image
lz4 -f Image Image.lz4
dd if=/dev/zero of=bootloader.bin bs=512 count=1
cp /compile/doc/stable/misc.cbq/misc/cmdline cmdline
mkimage -D "-I dts -O dtb -p 2048" -f auto -A arm64 -O linux -T kernel -C lz4 -a 0 -d Image.lz4 -b arch/arm64/boot/dts/qcom/sc7180-trogdor-coachz-r1-lte.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-coachz-r1.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-coachz-r3.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r0.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-coachz-r3-lte.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r1.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r1-kb.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r3.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r1-lte.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r3-kb.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r3-lte.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-limozeen.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-limozeen-nots.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-limozeen-nots-r4.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-pompom-r1.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-pompom-r2-lte.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-pompom-r1-lte.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-pompom-r2.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-pompom-r3-lte.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-pompom-r3.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-r1.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-r1-lte.dtb kernel.itb kernel.itb
vbutil_kernel --pack vmlinux.kpart --keyblock /usr/share/vboot/devkeys/kernel.keyblock --signprivate /usr/share/vboot/devkeys/kernel_data_key.vbprivk --version 1 --config cmdline --bootloader bootloader.bin --vmlinuz kernel.itb --arch arm
cp -v vmlinux.kpart /boot/vmlinux.kpart-${kver}
rm -f Image Image.lz4 cmdline bootloader.bin kernel.itb vmlinux.kpart
# end chromebook special

Further, I wonder if you have tried connecting a serial console to obtain the early kernel logs before the crash? I suppose a (DIY) SuzyQ could help with that? https://chromium.googlesource.com/chromiumos/third_party/hdctools/+/HEAD/docs/ccd.md#communicating-with-cr50 Specifically, adding earlycon kernel cmdline parameter should trigger the output to the serial interface. This may make debugging easier?

[Geofferey]

@leezu You might wanna check out (#7 in other repo). Do you have CONFIG_DRM_PANEL_EDP=y in defconfig? It's needed for any kernel past 5.15.X from what hexdump0815 said. I also posted a GIST that may aid you assuming you're on trogdor (coachz), if not it could easily be modified.

Here's the main line you would likely need to modify.... Already set for you in example below. Might need to remove missing DTBs.

 DT_BLOBS="-b arch/arm64/boot/dts/qcom/sc7180-trogdor-coachz-r1-lte.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-coachz-r1.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-coachz-r3.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r0.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-coachz-r3-lte.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r1.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r1-kb.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r3.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r1-lte.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r3-kb.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-r3-lte.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-limozeen.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-limozeen-nots.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-lazor-limozeen-nots-r4.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-pompom-r1.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-pompom-r2-lte.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-pompom-r1-lte.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-pompom-r2.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-pompom-r3-lte.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-pompom-r3.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-r1.dtb -b arch/arm64/boot/dts/qcom/sc7180-trogdor-r1-lte.dtb"

[hexdump0815]

one thing which comes to mind is that in case you are running with luks encryption on emmc then this section is very important (adapted to your system and kernel) and has to be run after each kernel installation as for luks based setup the initrd (which gets part of the kpart image) needs to be built with the proper crypttab setup, i.e. the initrd has to be updated for each system after it was built as part of the kernel build ... could it be that this step was missing? - this one here: https://github.com/hexdump0815/imagebuilder/blob/main/info/generic/install-to-emmc-with-luks-full-disk-encryption.txt#L196-L246

ps: plus the CONFIG_DRM_PANEL_EDP=y mentioned by @Geofferey above for all kernels > v5.15

[leezu]

Thank you @Geofferey and @hexdump0815! After adding CONFIG_DRM_PANEL_EDP=y, I'm able to boot the 5.18-rc5 kernel and most things work smoothly. I haven't identified any issue present on 5.15 that got solved on 5.18-rc5, but there may be some. The current status is documented at https://github.com/hexdump0815/imagebuilder/issues/47

One problem with the dd based installation and resizing is that it breaks systemd-gpt-auto-generator:

[    3.806937] systemd-gpt-auto-generator[243]: File system behind root file system is reported by btrfs to be backed by pseudo-device /dev/root, which is not a valid userspace accessible device node. Cannot determine correct backing block device.
[    3.894881] systemd[236]: /lib/systemd/system-generators/systemd-gpt-auto-generator failed with exit status 1.

@hexdump0815 did you use /dev/root during image generation?

one thing which comes to mind is that in case you are running with luks encryption on emmc then this section is very important (adapted to your system and kernel) and has to be run after each kernel installation as for luks based setup the initrd (which gets part of the kpart image) needs to be built with the proper crypttab setup, i.e. the initrd has to be updated for each system after it was built as part of the kernel build ... could it be that this step was missing? - this one here: https://github.com/hexdump0815/imagebuilder/blob/main/info/generic/install-to-emmc-with-luks-full-disk-encryption.txt#L196-L246

I did run the update-initramfs after specifying the crypttab and passed it to mkimage. I noticed that the output of mkimage lists the compression of the initrd as unknown, whereas the compression type of the kernel was correctly printed (based on whatever I pass via -C flag). Could that be an issue?

As a workaround for lack of full-disk encryption gocryptfs for the home directories can be an intermediate solution https://leighmcculloch.com/posts/ubuntu-encrypt-home-directory-with-gocryptfs/

[hexdump0815]

first a big thank you to @Geofferey , @leezu and the others - it is very nice to see some life, broader testing and good suggestions coming into the framework and the images

@leezu - i think i never explicitely used /dev/root anywhere - it must come in indirectly somewhere somehow - lets see if we can find out where ... i noticed the /dev/root a few times already as well in df, but as far as i remember did not yet run into any problems because of it

@leezu - regarding the initrd compression i think this should not be a problem as the initrd is not decompressed by the bootloader (the kernel is), but by the kernel which should know how to handle it due to some magic header in the compressed image, which is just handed over as a range in memory to the kernel as far as i know

[leezu]

@hexdump0815 I compared the GPT on your image against the one obtained by following the EMMC instructions via sfdisk -d /dev/mmcblk1 and sfdisk -d /dev/sda. Your image contains extra partition attributes attrs="GUID:49,51,53,56" and attrs="GUID:48,50,53". Manually setting them fixes the EMMC guide:

sfdisk --part-attrs /dev/mmcblk1 1 "GUID:49,51,53,56"
sfdisk --part-attrs /dev/mmcblk1 2 "GUID:48,50,53"

Are you aware? And do you know the meaning of this additional attribute? Without the attributes, Chromebook does not consider the partitions as valid boot partitions and does not even attempt to boot from them.

root@changeme:/boot# sfdisk -d /dev/mmcblk1
label: gpt
label-id: C7A22BBC-7B33-F540-8BFC-90F5E6FA6059
device: /dev/mmcblk1
unit: sectors
first-lba: 34
last-lba: 122142686
sector-size: 512

/dev/mmcblk1p1 : start=        8192, size=       65536, type=FE3A2A5D-4F32-41A7-B725-ACCC3285A309, uuid=56A13FC4-3F82-6440-BA81-919053F6E0ED, name="KernelA", attrs="GUID:49,51,53,56"
/dev/mmcblk1p2 : start=       73728, size=       65536, type=FE3A2A5D-4F32-41A7-B725-ACCC3285A309, uuid=E9DDC350-73C2-C244-8FC3-7E86A605B03F, name="KernelB", attrs="GUID:48,50,53"
/dev/mmcblk1p3 : start=      139264, size=     1048576, type=0FC63DAF-8483-4772-8E79-3D69D8477DE4, uuid=D314A820-1AE6-4BB7-B48A-5F54DF347DAD, name="primary", attrs="LegacyBIOSBootable"
/dev/mmcblk1p4 : start=     1187840, size=   120952832, type=0FC63DAF-8483-4772-8E79-3D69D8477DE4, uuid=DCDA9D97-6CF6-4782-BAEA-3888B22731F2, name="primary"

Finally, booting from encrypted disk via initramfs I noticed that kernel module would no longer be loaded due to LoadPin. Everything would work well after adding loadpin.enforce=0 to the kernel boot parameters. Do you know why LoadPin is being enabled by default when following the emmc guide / when initramfs is used?

[hexdump0815]

@leezu - thanks a lot for this information

with "emmc instructions" you mean https://github.com/hexdump0815/imagebuilder/blob/main/info/generic/install-to-emmc-with-luks-full-disk-encryption.txt - right? i can just say, that i did set up quite a few different chromebooks following those instructions and it always worked ... but i did not do it yet on a trogdor device which is what you are using (right?) - so maybe there is something different for that one? (but i'm not really sure about this) ... i also did not yet notice that loadpin is enabled by default - it could be that the depthcharge bootloader adds it by default maybe? i'll have to look deeper at this at some point and thus will put it onto the todo list

[leezu]

@hexdump0815 Yes, I'm on Trogdor (https://acerrecertified.com/acer-spin-513-13-3-chromebook-qualcomm-7c-2-1ghz-4gb-ram-64gb-flash-chromeos-cp513-1h-s60f/). I'm using your Debian image to perform the installation, but then switch to the debian testing (bookworm) channel. It's possible Loadpin is related to bookworm version. But Loadpin is not my major concern. Rather, I wonder why your images come with "GUID:49,51,53,56" and "GUID:48,50,53" GPT attributes for boot partition 1 and 2 respectively. If you know why, it would be great if you can shed some light on it. If not, let's at least modify the emmc guide to explicitly document that the attributes must be set (at least for Trogdor). WDYT?

[hexdump0815]

@leezu - this is really strange: i'm using the exact same command in the emmc install notes - https://github.com/hexdump0815/imagebuilder/blob/main/info/generic/install-to-emmc-with-luks-full-disk-encryption.txt#L38-L39 as i do during image creation - https://github.com/hexdump0815/imagebuilder/blob/main/scripts/create-image.sh#L162-L163 ... i'll really have to check this, but sadly do not have any trogdor device at hand right now, but that should change soon ... then i'll definitely check and if required update it accordingly for both the partition settings and the kernel cmdline options

[crazycusti]

I'm not sure if this will apply to you but I took a much simpler approach for a arm64 device (trogdor). Write the image to USB drive and store a copy of that same image you used to flash drive in changeme's home dir on USB drive. Boot the drive on your device and write the image to /dev/mmcblk1 (in my case) using dd. If you can't find a way to get image on drive before booting put it on another and plug that it in while booted from sd/USB to have img available for writing.

dd if=~/chromebook-linux.img of=/dev/mmcblk1

Writing the image this way should partition the drive and everything so you don't need to do so manually. Afterwards make sure to remove the USB before next boot and expand volume etc on first boot from emmc using /scripts/extend-rootfs.sh. Be sure to remove USB/SD before booting from internel or it may use the data located on your initial media source.

I can not guarantee this will work for you, nor can I guarantee it's appropriate for your device. It did work for me and my Chromebook seems to be running just fine. YMMV.

thanks! it work on my kukui based chromebook!

[thomas725]

I'd like to try the luks emmc installation and found the path to the guide has changed. The above links give 404 errors. It now resides at: https://github.com/hexdump0815/imagebuilder/blob/main/doc/install-to-emmc-with-luks-full-disk-encryption.txt And it seems this markdown file also describes the process but is easier to read: https://github.com/hexdump0815/imagebuilder/blob/main/doc/install-to-emmc-on-arm-chromebooks.md

[hexdump0815]

the other file is still around as well as https://github.com/hexdump0815/imagebuilder/blob/main/doc/install-to-emmc-with-luks-full-disk-encryption.txt - there was interest in easier to read and to understand instructions for chromebook emmc installation and as a result i created the newer md file which only covers the chromebook case and does not have the luks encrypted option in it yet