milas / talos-sbc-rk3588

System overlay & custom kernel/installer to support running Talos Linux on RK3588
Mozilla Public License 2.0
9 stars 4 forks source link

Assistance with Creating a Bootable Image (.raw) #5

Open Christos822 opened 5 months ago

Christos822 commented 5 months ago

Hello @milas

Could you please describe how we create the bootable image (.raw) ? I tried by running the following : make target-talos-sbc-rk3588 PLATFORM=linux/arm64 USERNAME=christos822 PUSH=true, However, after this step, I am unsure how to proceed to create the .raw image. Could you please provide guidance on the next steps?? I would really appreciate your assistance.

Best Regards, Christos

PS I managed to implement Talos 1.4.6 for Radxa CM3 board. The initial errors I encountered during booting were resolved by correcting the dtb file I was using.

milas commented 5 months ago

Hi Christos,

(I'm so glad you were able to get the previous version working for the CM3!)

The best reference for now is the GitHub Actions workflow: https://github.com/milas/talos-sbc-rk3588/blob/main/.github/workflows/ci.yaml

Also useful:

It's a little bit involved 😅 but here's the main steps:

  1. make target-talos-sbc-rk3588 : overlay for RK3588
  2. make target-talos-kernel-rk3588 : kernel for RK3588
  3. Clone upstream siderolabs/talos repo
  4. Modify hack/modules-arm64.txt in the Talos repo based on your kernel
  5. In Talos repo, make imager PKG_KERNEL=<image from step (2)>
  6. Run your imager image to produce a bootable .raw.xz image
    docker run --rm -t -v ./_out:/out -v /dev:/dev --privileged <imager image from step (5)> \
    installer --arch arm64 \
      --base-installer-image="ghcr.io/siderolabs/installer:v1.7.4" \
      --overlay-name=rk3588 \
      --overlay-image=<overlay image from step (1)> \
      --overlay-option="board=<your board>" \
      --overlay-option="chipset=<rk3588/rk3588s>" \
      --system-extension-image=<realtek firmware etc>
  7. Push the installer image
    crane push _out/installer-arm64.tar <registry ref>
  8. Run the imager to produce a flashable image using the installer
    docker run --rm -t -v ./_out:/out -v /dev:/dev --privileged <imager image from step (5)> \
    metal --arch arm64 \
    --overlay-image=<overlay image from step (1)> \
    --overlay-name=rk3588 \
    --overlay-option="board=<your board>" \
    --overlay-option="chipset=<rk3588/rk3588s>" \
    --base-installer-image=<installer image from (7)>

For reference, here's the latest CI job for the Rock 5A: https://github.com/milas/talos-sbc-rk3588/actions/runs/9323195360/job/25665933453

That ends up being:

# build overlay + kernel [this repo]
make PUSH=true
# build Talos imager with custom kernel
git clone https://github.com/siderolabs/talos.git
cd ./talos
git am --whitespace=fix ../hack/patches/talos/0001-initramfs-kernel-modules.patch
make imager \
  PKG_KERNEL="ghcr.io/milas/talos-kernel-rk3588:v1.7.4-rk3588.alpha.5" \
  TAG=v1.7.4-rk3588.alpha.5 \
  PUSH=true
# create installer image
docker run --rm -t -v ./_out:/out -v /dev:/dev --privileged ghcr.io/milas/imager:v1.7.4-rk3588.alpha.5 \
  installer --arch arm64 \
    --base-installer-image="ghcr.io/siderolabs/installer:v1.7.4" \
    --overlay-name=rk3588 \
    --overlay-image=ghcr.io/milas/talos-sbc-rk3588:v1.7.4-rk3588.alpha.5 \
    --overlay-option="board=rock-5a" \
    --overlay-option="chipset=rk3588s" \
    --system-extension-image="ghcr.io/siderolabs/realtek-firmware:20240513@sha256:4ca40c2836c1cdb5105456186afd880925d72e81ee6b0ff69a40c9c05b7b74a4"
# push installer image
crane push _out/installer-arm64.tar ghcr.io/milas/talos-rk3588:v1.7.4-rk3588.alpha.5-rock-5a
# create flashable image
docker run --rm -t -v ./_out:/out -v /dev:/dev --privileged ghcr.io/milas/imager:v1.7.4-rk3588.alpha.5 \
metal --arch arm64 \
  --overlay-image=ghcr.io/milas/talos-sbc-rk3588:v1.7.4-rk3588.alpha.5 \
  --overlay-name=rk3588 \
  --overlay-option="board=rock-5a" \
  --overlay-option="chipset=rk3588s" \
  --base-installer-image="ghcr.io/milas/talos-rk3588:v1.7.4-rk3588.alpha.5-rock-5a"
Christos822 commented 5 months ago

Hi @milas,

Thank you very much for the detailed description.

I managed to make a bootable image for the CM3 board, but unfortunately, only the bootloader activates. I used the u-boot.img, as I have done in the previous Talos image (1.4.6), with the following changes in main.go:

  1. I updated the path for reading the u-boot image to match the new location: `uboot, err := os.ReadFile(filepath.Join(options.ArtifactsPath, fmt.Sprintf("arm64/u-boot/%s/u-boot.img", options.ExtraOptions.Board))) // Old line for reference // uboot, err := os.ReadFile(filepath.Join(options.ArtifactsPath, fmt.Sprintf("arm64/u-boot/%s/u-boot-rockchip.bin", options.ExtraOptions.Board)))

  2. I modified the PartitionOptions to change the offset, which previously resulted in an image with only one partition:

`PartitionOptions: overlay.PartitionOptions{ // Old offset for reference // Offset: 2048 * 10, Offset: 32768, }

3.Adjusted the way u-boot is written to the file:

`uboot = uboot[ubootOffset:] if _, err = f.WriteAt(uboot, ubootOffset); err != nil { return fmt.Errorf("writing u-boot: %w", err) }

Additionally, I used the vmlinuz and modules of the 5.10 kernel , those that worked in the 1.4.6 version , to test, but without success. I also tried building and using the 6.6.2 kernel from the Radxa BSP, but no results either. Before wrapping up, I wanted to ask if the latest Talos requires the latest kernel version as well? I'm wondering if that might be part of the issue. Thank you once again for your great work and support.

Best, Christos