Open rNoz opened 3 weeks ago
Hello @rNoz
Thanks for your feedback, we will check this issue.
Hi @rNoz
Here is the list of modules needed for docker. You can create /etc/modules-load.d/docker.conf
with the following content. Docker should start working after reboot.
nfnetlink
nf_tables
nft_counter
nft_objref
nft_compat
nf_defrag_ipv4
nf_defrag_ipv6
nf_conntrack
nf_nat
nft_chain_nat
xt_MASQUERADE
xt_addrtype
xt_conntrack
xt_tcpudp
xt_nat
I am also checking on microk8s and k3s and will shortly revert back on the same.
List of kernel modules needed for microk8s
nfnetlink
nf_tables
xt_comment
xt_mark
nf_defrag_ipv4
nf_defrag_ipv6
nf_conntrack
nf_nat
iptable_nat
iptable_mangle
iptable_filter
nf_reject_ipv4
ipt_REJECT
ip6_tables
ip6table_nat
ip6table_mangle
ip6table_filter
nf_reject_ipv6
ip6t_REJECT
Raised https://github.com/khadas/fenix/pull/300 to load the modules by default.
Thank you!
@viraniac Any idea why snapd does not work right now for Vim4 in the latest Ubuntu 24.04?
k@Khadas:~$ sudo apt update && sudo apt upgrade
# everything up to date
k@Khadas:~$ sudo snap remove microk8s
error: system does not fully support snapd: cannot mount squashfs image using
"squashfs": ----- mount: /tmp/syscheck-mountpoint-2162884218: wrong fs type,
bad option, bad superblock on /dev/loop0, missing codepage or helper program,
or other error.
dmesg(1) may have more information after failed mount system call.
-----
k@Khadas:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 24.04 LTS
Release: 24.04
Codename: noble
k@Khadas:~$ uname -a
Linux Khadas 5.15.119 #1.6.9.1 SMP PREEMPT Thu Jun 20 09:22:43 CST 2024 aarch64 aarch64 aarch64 GNU/Linux
/boot/uEnv.txt:
boot_user_args=cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory systemd.unified_cgroup_hierarchy=0
/etc/modules-load.d/microk8s.conf:
nfnetlink
nf_tables
xt_comment
xt_mark
nf_defrag_ipv4
nf_defrag_ipv6
nf_conntrack
nf_nat
iptable_nat
iptable_mangle
iptable_filter
nf_reject_ipv4
ipt_REJECT
ip6_tables
ip6table_nat
ip6table_mangle
ip6table_filter
nf_reject_ipv6
ip6t_REJECT
k@Khadas:~$ dpkg -l | grep -i -E '(khadas|vim|linux|snap)' | grep -E '^ii\s*(khadas|linux|snap|libsnap)'
ii khadas-vim4-linux-5.4-dt-overlays 1.6.5~2337ee9 arm64 Linux DTB overlays.
ii libsnapd-glib-2-1:arm64 1.64-0ubuntu5 arm64 GLib snapd library
ii libsnappy1v5:arm64 1.1.10-1build1 arm64 fast compression/decompression library
ii linux-base 4.5ubuntu9 all Linux image base package
ii linux-board-package-noble-vim4 1.6.9 arm64 Fenix tweaks for Ubuntu-noble on VIM4
ii linux-dtb-amlogic-5.15 1.6.9.1 arm64 Linux DTB, version 5.15.119
ii linux-gpu-mali-wayland 1.4-r44p0-202311 arm64 Lib Mali Wayland for A311D2
ii linux-headers-amlogic-5.15 1.6.9.1 arm64 Linux kernel headers for 5.15.119 on arm64
ii linux-image-amlogic-5.15 1.6.9.1 arm64 Linux kernel, version 5.15.119
ii linux-libc-dev:arm64 6.8.0-38.38 arm64 Linux Kernel Headers for development
ii linux-sound-base 1.0.25+dfsg-0ubuntu7 all base package for ALSA and OSS sound systems
ii linux-u-boot-vim4-vendor 1.6.9-2019.01 arm64 U-boot 2019.01
ii snapd 2.63+24.04 arm64 Daemon and tooling that enable snap packages
No matter the command (remove, install) or application.
Writing here because the PR 300 was merged.
Writing here because the PR 300 was merged.
The changes in the #300 are merged, but was not part of last image release and should get included in the next release.
I see you are trying to remove microk8s. Did you had it installed before? Does snap list
show any installed snap packages? By default no snap packages are included in the image, and they get installed as needed when user runs the snap install command
@rNoz I am trying to replicate the issue. Will get back to you with my findings.
@rNoz I can't replicate your issue. Please find my output here - https://paste.armbian.com/paxixiquyo.bash
What I have tried:
sudo apt-get update && sudo apt-get dist-upgrade && sudo reboot
wget -O- https://raw.githubusercontent.com/khadas/fenix/master/archives/filesystem/special/VIM-COMMON/etc/modules-load.d/docker.conf | sudo tee /etc/modules-load.d/docker.conf
wget -O- https://raw.githubusercontent.com/khadas/fenix/master/archives/filesystem/special/VIM-COMMON/etc/modules-load.d/microk8s.conf | sudo tee /etc/modules-load.d/microk8s.conf
sudo snap install microk8s --channel=1.28/stable --classic
sudo usermod -aG microk8s khadas
newgrp microk8s
microk8s disable ha-cluster
microk8s enable dashboard
microk8s enable dns
microk8s enable hostpath-storage
microk8s kubectl get all --all-namespaces
I have the same modules and everything up to date, but snap (not necessarily for microk8s) is not working due to the squashfs error.
k@Khadas:~$ sudo snap list
No snaps are installed yet. Try 'snap install hello-world'.
k@Khadas:~$ sudo snap install hello-world
error: system does not fully support snapd: cannot mount squashfs image using "squashfs": -----
mount: /tmp/syscheck-mountpoint-1357931385: wrong fs type, bad option, bad superblock on
/dev/loop0, missing codepage or helper program, or other error.
dmesg(1) may have more information after failed mount system call.
-----
journal
Jul 15 10:41:45 Khadas snapd[2199]: daemon.go:519: gracefully waiting for running hooks
Jul 15 10:41:45 Khadas snapd[2199]: daemon.go:521: done waiting for running hooks
Jul 15 10:41:48 Khadas snapd[2199]: overlord.go:515: Released state lock file
Jul 15 10:41:48 Khadas snapd[2199]: daemon stop requested to wait for socket activation
Jul 15 10:41:48 Khadas systemd[1]: snapd.service: Deactivated successfully.
Jul 15 10:42:36 Khadas systemd[1]: Starting snapd.service - Snap Daemon...
Jul 15 10:42:36 Khadas snapd[2276]: overlord.go:271: Acquiring state lock file
Jul 15 10:42:36 Khadas snapd[2276]: overlord.go:276: Acquired state lock file
Jul 15 10:42:36 Khadas snapd[2276]: daemon.go:247: started snapd/2.63+24.04 (series 16; classic; devmode) ubuntu/24.04 (arm64) linux/5.15.119.
Jul 15 10:42:36 Khadas snapd[2276]: main.go:125: system does not fully support snapd: cannot mount squashfs image using "squashfs":
Jul 15 10:42:36 Khadas snapd[2276]: -----
Jul 15 10:42:36 Khadas snapd[2276]: mount: /tmp/syscheck-mountpoint-787852083: wrong fs type, bad option, bad superblock on /dev/loop0, missing codepage or helper program, or other error.
Jul 15 10:42:36 Khadas snapd[2276]: dmesg(1) may have more information after failed mount system call.
Jul 15 10:42:36 Khadas snapd[2276]: -----
Jul 15 10:42:36 Khadas snapd[2276]: daemon.go:340: adjusting startup timeout by 30s (pessimistic estimate of 30s plus 5s per snap)
Jul 15 10:42:36 Khadas snapd[2276]: backends.go:58: AppArmor status: apparmor not enabled
Jul 15 10:42:36 Khadas systemd[1]: Started snapd.service - Snap Daemon.
Jul 15 10:42:41 Khadas snapd[2276]: daemon.go:519: gracefully waiting for running hooks
Jul 15 10:42:41 Khadas snapd[2276]: daemon.go:521: done waiting for running hooks
Jul 15 10:42:44 Khadas snapd[2276]: overlord.go:515: Released state lock file
Jul 15 10:42:44 Khadas snapd[2276]: daemon stop requested to wait for socket activation
Jul 15 10:42:44 Khadas systemd[1]: snapd.service: Deactivated successfully.
dmesg
[ 43.091529] loop0: detected capacity change from 0 to 8
[ 43.092295] SELinux: security_context_str_to_sid(system_u:object_r:snappy_snap_t:s0) failed for (dev ?, type ?) errno=-22
[ 43.093260] SELinux: security_context_str_to_sid(system_u:object_r:snappy_snap_t:s0) failed for (dev loop0, type squashfs) errno=-22
[ 51.966584] loop0: detected capacity change from 0 to 8
[ 51.968003] SELinux: security_context_str_to_sid(system_u:object_r:snappy_snap_t:s0) failed for (dev ?, type ?) errno=-22
[ 51.968817] SELinux: security_context_str_to_sid(system_u:object_r:snappy_snap_t:s0) failed for (dev loop0, type squashfs) errno=-22
[ 86.892334] audit: type=1107 audit(1721040135.261:12): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:kernel_t:s0 msg='avc: denied { status } for auid=n/a uid=0 gid=0 path="/usr/lib/systemd/system/snapd.service" cmdline="" function="mac_selinux_filter" scontext=system_u:system_r:kernel_t:s0 tcontext=system_u:object_r:systemd_unit_t:s0 tclass=service permissive=1 exe="/usr/lib/systemd/systemd" sauid=0 hostname=? addr=? terminal=?'
[ 108.174239] loop0: detected capacity change from 0 to 8
[ 108.174950] SELinux: security_context_str_to_sid(system_u:object_r:snappy_snap_t:s0) failed for (dev ?, type ?) errno=-22
[ 108.175399] SELinux: security_context_str_to_sid(system_u:object_r:snappy_snap_t:s0) failed for (dev loop0, type squashfs) errno=-22
Do I have to do anything with SELinux? I am not sure why snapd stopped working
Indeed, just by restarting snapd
, I get that error:
Jul 15 10:58:14 Khadas systemd[1]: snapd.service: Deactivated successfully.
Jul 15 10:58:17 Khadas systemd[1]: Starting snapd.service - Snap Daemon...
Jul 15 10:58:17 Khadas snapd[5958]: overlord.go:271: Acquiring state lock file
Jul 15 10:58:17 Khadas snapd[5958]: overlord.go:276: Acquired state lock file
Jul 15 10:58:17 Khadas snapd[5958]: daemon.go:247: started snapd/2.63+24.04 (series 16; classic; devmode) ubuntu/24.04 (arm64) linux/5.15.119.
Jul 15 10:58:17 Khadas snapd[5958]: main.go:125: system does not fully support snapd: cannot mount squashfs image using "squashfs":
Jul 15 10:58:17 Khadas snapd[5958]: -----
Jul 15 10:58:17 Khadas snapd[5958]: mount: /tmp/syscheck-mountpoint-1668328185: wrong fs type, bad option, bad superblock on /dev/loop0, missing codepage or helper program, or other error.
Jul 15 10:58:17 Khadas snapd[5958]: dmesg(1) may have more information after failed mount system call.
Jul 15 10:58:17 Khadas snapd[5958]: -----
Jul 15 10:58:17 Khadas snapd[5958]: daemon.go:340: adjusting startup timeout by 30s (pessimistic estimate of 30s plus 5s per snap)
Jul 15 10:58:17 Khadas snapd[5958]: backends.go:58: AppArmor status: apparmor not enabled
Jul 15 10:58:17 Khadas systemd[1]: Started snapd.service - Snap Daemon.
I have already tried installing sudo apt install libsquashfuse0 squashfuse fuse squash squashfs-tools
, but the same error.
More trials:
k@Khadas:~$ sudo apt install selinux-policy-default
# already installed
k@Khadas:~$ sestatus
SELinux status: enabled
SELinuxfs mount: /sys/fs/selinux
SELinux root directory: /etc/selinux
Loaded policy name: default
Current mode: permissive
Mode from config file: permissive
Policy MLS status: enabled
Policy deny_unknown status: allowed
Memory protection checking: actual (secure)
Max kernel policy version: 33
k@Khadas:~$ sudo setenforce 0
Ok, Steps:
sudo apt remove snapd
reboot
sudo apt install snapd
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages were automatically installed and are no longer required:
gir1.2-gnomeautoar-0.1 gir1.2-gnomedesktop-3.0 gjs libcue2 libdee-1.0-4 libexempi8
libgexiv2-2 libgsf-1-114 libgsf-1-common libnautilus-extension4 libntfs-3g89t64
libportal-gtk4-1 libportal1 libtotem-plparser-common libtotem-plparser18
libtracker-sparql-3.0-0 libunity-protocol-private0
libunity-scopes-json-def-desktop libunity9 nautilus-data tracker tracker-extract
tracker-miner-fs xwayland
Use 'sudo apt autoremove' to remove them.
The following NEW packages will be installed:
snapd
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 25.2 MB of archives.
After this operation, 106 MB of additional disk space will be used.
Get:1 http://ports.ubuntu.com noble-updates/main arm64 snapd arm64 2.63+24.04 [25.2 MB]
Fetched 25.2 MB in 11s (2,327 kB/s)
Selecting previously unselected package snapd.
(Reading database ... 150033 files and directories currently installed.)
Preparing to unpack .../snapd_2.63+24.04_arm64.deb ...
Unpacking snapd (2.63+24.04) ...
Setting up snapd (2.63+24.04) ...
snapd.failure.service is a disabled or a static unit not running, not starting it.
snapd.snap-repair.service is a disabled or a static unit not running, not starting it.
Failed to restart snapd.mounts-pre.target: Operation refused, unit snapd.mounts-pre.target may be requested by dependency only (it is configured to refuse manual start/stop).
See system logs and 'systemctl status snapd.mounts-pre.target' for details.
Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 148.
Processing triggers for gnome-menus (3.36.0-1.1ubuntu3) ...
Processing triggers for man-db (2.12.0-4build2) ...
Processing triggers for dbus (1.14.10-4ubuntu4) ...
Processing triggers for desktop-file-utils (0.27-2build1) ...
dmesg:
[ 81.656526] audit: type=1400 audit(1721041718.739:12): avc: denied { transition } for pid=2131 comm="dpkg" path="/var/lib/dpkg/tmp.ci/preinst" dev="mmcblk0p5" ino=386038 scontext=system_u:system_r:kernel_t:s0 tcontext=system_u:system_r:dpkg_script_t:s0 tclass=process permissive=1
[ 81.656566] audit: type=1400 audit(1721041718.739:13): avc: denied { entrypoint } for pid=2131 comm="dpkg" path="/var/lib/dpkg/tmp.ci/preinst" dev="mmcblk0p5" ino=386038 scontext=system_u:system_r:dpkg_script_t:s0 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file permissive=1
[ 89.009671] loop0: detected capacity change from 0 to 8
[ 89.010167] SELinux: security_context_str_to_sid(system_u:object_r:snappy_snap_t:s0) failed for (dev ?, type ?) errno=-22
[ 89.010487] SELinux: security_context_str_to_sid(system_u:object_r:snappy_snap_t:s0) failed for (dev loop0, type squashfs) errno=-22
[ 90.543760] audit: type=1400 audit(1721041727.627:14): avc: denied { entrypoint } for pid=2461 comm="dpkg" path="/var/lib/dpkg/info/man-db.postinst" dev="mmcblk0p5" ino=185094 scontext=system_u:system_r:dpkg_script_t:s0 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file permissive=1
[ 91.561867] audit: type=1107 audit(1721041728.643:15): pid=904 uid=101 auid=4294967295 ses=4294967295 subj=system_u:system_r:kernel_t:s0 msg='avc: denied { send_msg } for msgtype=method_call interface=org.freedesktop.DBus member=Hello dest=org.freedesktop.DBus spid=2464 scontext=system_u:system_r:dpkg_script_t:s0 tcontext=system_u:system_r:kernel_t:s0 tclass=dbus permissive=1 exe="/usr/bin/dbus-daemon" sauid=101 hostname=? addr=? terminal=?'
Indeed, just by restarting
snapd
, I get that error:
Tried restarting snapd, I don't get any errors
khadas@Khadas:~$ sudo systemctl status
[sudo] password for khadas:
sudo: a password is required
khadas@Khadas:~$ sudo systemctl status snapd
[sudo] password for khadas:
● snapd.service - Snap Daemon
Loaded: loaded (/usr/lib/systemd/system/snapd.service; enabled; preset: enabled)
Active: active (running) since Mon 2024-07-15 11:07:30 UTC; 1min 11s ago
TriggeredBy: ● snapd.socket
Main PID: 613285 (snapd)
Tasks: 15 (limit: 7100)
Memory: 13.1M ()
CPU: 3.323s
CGroup: /system.slice/snapd.service
└─613285 /usr/lib/snapd/snapd
Jul 15 11:07:30 Khadas systemd[1]: Starting snapd.service - Snap Daemon...
Jul 15 11:07:30 Khadas snapd[613285]: overlord.go:271: Acquiring state lock file
Jul 15 11:07:30 Khadas snapd[613285]: overlord.go:276: Acquired state lock file
Jul 15 11:07:30 Khadas snapd[613285]: daemon.go:247: started snapd/2.63+24.04 (series 16; classic; devmode) ubuntu/24.04 (arm64) linux/5.15.119.
Jul 15 11:07:30 Khadas snapd[613285]: daemon.go:340: adjusting startup timeout by 45s (pessimistic estimate of 30s plus 5s per snap)
Jul 15 11:07:30 Khadas snapd[613285]: backends.go:58: AppArmor status: apparmor not enabled
Jul 15 11:07:30 Khadas systemd[1]: Started snapd.service - Snap Daemon.
@rNoz I am struggling to reproduce your issue. Have you made any other changes? Can you try once on a fresh installation?
I have no squashfs related errors. Mount command shows snaps mounted using squashfs as well.
khadas@Khadas:~$ mount | grep squash
/var/lib/snapd/snaps/snapd_21761.snap on /snap/snapd/21761 type squashfs (ro,nodev,relatime,errors=continue,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/core20_2321.snap on /snap/core20/2321 type squashfs (ro,nodev,relatime,errors=continue,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/microk8s_6893.snap on /snap/microk8s/6893 type squashfs (ro,nodev,relatime,errors=continue,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/core22_1383.snap on /snap/core22/1383 type squashfs (ro,nodev,relatime,errors=continue,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/bare_5.snap on /snap/bare/5 type squashfs (ro,nodev,relatime,errors=continue,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/chromium_2899.snap on /snap/chromium/2899 type squashfs (ro,nodev,relatime,errors=continue,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/cups_1059.snap on /snap/cups/1059 type squashfs (ro,nodev,relatime,errors=continue,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/gtk-common-themes_1535.snap on /snap/gtk-common-themes/1535 type squashfs (ro,nodev,relatime,errors=continue,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/gnome-42-2204_178.snap on /snap/gnome-42-2204/178 type squashfs (ro,nodev,relatime,errors=continue,x-gdu.hide,x-gvfs-hide)
I would like to solve this because I got this trouble in the past and a fresh install solved it, but with just a few weeks working with it, it appeared again. So, I would prefer to fix it, as it is not possible to always do a fresh install to fix it. In that case I won't use snapd, since it is unstable in khadas.
Any idea how can I go for it?
It is definitely related with SELinux and a kernel version.
# edit /etc/selinux/config
SELINUX=disabled
Then reboot.
Now it works. No errors. microk8s
is installed and works successfully.
So, any idea how we can make snapd working with the default policy for selinux (SELINUX=permissive) for khadas vim3/4?
So, any idea how we can make snapd working with the default policy for selinux (SELINUX=permissive) for khadas vim3/4?
AFAIK, there is no selinux policy installed by default and with permissive selinux should allow everything. I do find it weird that selinux is enabled as default security on ubuntu image. I only see libselinux installed on my system in the dpkg output. Have you installed any other selinux packages?
So, any idea how we can make snapd working with the default policy for selinux (SELINUX=permissive) for khadas vim3/4?
Either you can add the required policy using audit2allow. Or if selinux is not required add security=apparmor to your kernel commandline to switch to using apparmor by default
Detected, somehow at some point I installed selinux-policy-default
, introducing this problem. I have removed it (fully, also with dpkg --purge
to solve it). And I have added to uEnv.txt apparmor=1
.
At least it is registered in this issue. I think we can close it :)
Thanks for your time ;)
Hi,
All related to 22.04 and 24.04. The images that you have posted, tried both with the Dec 2023 version and the June 2024 versions of ubuntu gnome directly in the eMMC.
I don't know why but it has been impossible to install
microk8s
, in that case, due to squashfs. I installed squashfs-tools and tried all things I found, but without luck.The same with k3s and docker and the networking. Yyes, I have the uEnv.txt with the cgroups stuff and I can build images, but as soon as I try to expose ports or connect pods, it fails. It is all related to nf_tables/iptables. I tried update-alternatives legacy and so on, also loading manually the modules.
In the end, I always have errors like this:
(k3s)
(docker)
Here even forcing to load those modules.
So, probably it is something wrong with the ubuntu fenix kernel. What can I do?