Quartz64 | BPF syscalls and cgroup missing, breaking Docker

[am9zZWY]

Details:

• Date | Wed Nov 16 20:25:46 GMT 2022
• Bug report | 938ea672-2fab-47c6-a6c9-09828c0d66b6
• DietPi version | v8.10.2 (MichaIng/master)
• Image creator | DietPi Core Team
• Pre-image | from scratch
• Hardware | Quartz64 (aarch64) (ID=49)
• Kernel version | Linux DietPi 6.1.0-rc1 #1 SMP PREEMPT Wed Nov 9 01:08:14 UTC 2022 aarch64 GNU/Linux
• Distro | bullseye (ID=6)
• Command | docker run -d -p 9002:9000 --name=portainer --restart=always -v /run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer-ce
• Exit code | 125
• Software title | DietPi-Software

  Steps to reproduce:

  1. Run dietpi-software
  2. Select Portainer as software to be installed

     Expected behaviour:

• It should install Portainer

  Actual behaviour:

• It fails due to docker

  Extra details:

• I went through the forums and issues to see if anybody had similar issues. A solution was "always" to add systemd.unified_cgroup_hierarchy=0 to some /boot/boot.ini file, which I don't have.
• I installed Docker and docker-compose via dietpi-software and later I reinstalled it manually but it didn't solve any issue

  Additional logs:

  Unable to find image 'portainer/portainer-ce:latest' locally
  latest: Pulling from portainer/portainer-ce
  772227786281: Pulling fs layer
  96fd13befc87: Pulling fs layer
  4a224b6ae0d2: Pulling fs layer
  9943345fdfad: Pulling fs layer
  96fd13befc87: Download complete
  772227786281: Verifying Checksum
  772227786281: Download complete
  772227786281: Pull complete
  96fd13befc87: Pull complete
  9943345fdfad: Download complete
  4a224b6ae0d2: Verifying Checksum
  4a224b6ae0d2: Download complete
  4a224b6ae0d2: Pull complete
  9943345fdfad: Pull complete
  Digest: sha256:4bbfaf39eff84e9d2a84c988918fceaa6910a8c5095acf124a328a74b7cc1d6e
  Status: Downloaded newer image for portainer/portainer-ce:latest
  58b7c36df1b1230f27218ef571da67382c7f5703f012369bc8e8ad926284484a
  docker: Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error setting cgroup config for procHooks process: bpf_prog_query(BPF_CGROUP_DEVICE) failed: function not implemented: unknown.

root@DietPi:/boot# ls -lrt
total 35920
-rw-r--r-- 1 root root    15251 Oct 22 23:50 dietpi-README.md
-rw-r--r-- 1 root root    18092 Oct 22 23:50 dietpi-LICENSE.txt
-rw-r--r-- 1 root root 36657664 Nov 12 20:48 vmlinuz
-rw-r--r-- 1 root root    54806 Nov 12 20:48 rk3566-quartz64-a.dtb
drwxr-xr-x 2 root root     4096 Nov 12 21:11 u-boot
drwxr-xr-x 2 root root     4096 Nov 12 21:11 extlinux
drwxr-xr-x 4 root root     4096 Nov 16 19:50 dietpi
-rw-r--r-- 1 root root    16013 Nov 16 20:22 dietpi.txt

[Joulinar]

might be similar to this one https://github.com/MichaIng/DietPi/issues/5886

[MichaIng]

Probably not exactly the same, but of similar kind, actually more similar to: #5554

Does it work to append systemd.unified_cgroup_hierarchy=0 to the last line of /boot/extlinux/extlinux.conf?

[MichaIng]

MR up: https://gitlab.com/pgwipeout/quartz64_ci/-/merge_requests/5 But Peter hasn't been responsive to my other MR's so far, so no guarantee 😉.

[MichaIng]

Do I have to restart the system after the change?

yes

[am9zZWY]

No, it didn't help

[MichaIng]

Can you show the file content:

cat /boot/extlinux/extlinux.conf

And is the error message the same, or a different one?

[am9zZWY]

root@DietPi:~# cat /boot/extlinux/extlinux.conf
default l0
menu title Quartz64 boot menu
prompt 0
timeout 10

label l0
menu label Boot primary OS
linux /boot/vmlinuz
fdt /boot/rk3566-quartz64-a.dtb
append earlycon=uart8250,mmio32,0xfe660000 console=ttyS2,1500000n8 console=tty1 root=PARTUUID=e28ca6fa-01 rootfstype=ext4 rootwait fsck.repair=yes net.ifnames=0
systemd.unified_cgroup_hierarchy=0

The error message is the same

[MichaIng]

The settings needs to be "appended to the last line" and not added as new line 😉:

default l0
menu title Quartz64 boot menu
prompt 0
timeout 10

label l0
menu label Boot primary OS
linux /boot/vmlinuz
fdt /boot/rk3566-quartz64-a.dtb
append earlycon=uart8250,mmio32,0xfe660000 console=ttyS2,1500000n8 console=tty1 root=PARTUUID=e28ca6fa-01 rootfstype=ext4 rootwait fsck.repair=yes net.ifnames=0 systemd.unified_cgroup_hierarchy=0

[am9zZWY]

Omg! Yeah it works 🎉

[MichaIng]

Okay great. So same as with Tinker Board, we can either add this OOTB to both boards' boot settings, or we add it whenever Docker or K3s is installed. We already add this when a too low kernel version is detected, so this seems to be a good hook to minimise possible other negative impacts of forcing legacy cgroupv1.

[MichaIng]

Solved with, until the missing module is available: https://github.com/MichaIng/DietPi/commit/e41cd73

[MichaIng]

Better approach: https://github.com/MichaIng/DietPi/commit/3248247

[MichaIng]

Now solving this really.

[MichaIng]

Our new set of images has the needed kernel features now included, so that disabling cgroupv2 is not needed anymore: https://dietpi.com/downloads/images/testing/

The kernel itself can be upgraded as well with the matching firmware package from here: https://dietpi.com/downloads/binaries/

Will ship this upgrade with next DietPi update, after some more testing and probably little enhancements to the kernel config.