Closed bhargavthriler closed 2 years ago
tl;dr: PACKAGECONFIG:append:pn-kmod = " xz"
While I can't answer your question regarding the specific commit which introduced the changes to the kernel I might be able to help you loading the kernel modules.
I was in a similar situation and no kernel modules had been loaded after building a new image with latest openembedded-core, meta-openembedded, and meta-raspberrypi versions.
In my case there was no output on my screen but the root cause was the missing kernel modules.
Analysis
root@raspberrypi4-64:~# lsmod
Module Size Used by
root@raspberrypi4-64:~# find /lib/modules/5.15.34-v8/kernel -type f | head
/lib/modules/5.15.34-v8/kernel/drivers/connector/cn.ko.xz
/lib/modules/5.15.34-v8/kernel/drivers/w1/masters/ds2490.ko.xz
/lib/modules/5.15.34-v8/kernel/drivers/w1/masters/w1-gpio.ko.xz
/lib/modules/5.15.34-v8/kernel/drivers/w1/masters/ds1wm.ko.xz
/lib/modules/5.15.34-v8/kernel/drivers/w1/masters/ds2482.ko.xz
/lib/modules/5.15.34-v8/kernel/drivers/w1/slaves/w1_ds2438.ko.xz
/lib/modules/5.15.34-v8/kernel/drivers/w1/slaves/w1_ds28e04.ko.xz
/lib/modules/5.15.34-v8/kernel/drivers/w1/slaves/w1_ds2431.ko.xz
/lib/modules/5.15.34-v8/kernel/drivers/w1/slaves/w1_smem.ko.xz
/lib/modules/5.15.34-v8/kernel/drivers/w1/slaves/w1_ds2781.ko.xz
root@raspberrypi4-64:~# kmod --version
kmod version 29
-ZSTD -XZ +ZLIB -LIBCRYPTO -EXPERIMENTAL
root@raspberrypi4-64:~# which kmod
/bin/kmod
Notice that XZ is disabled for kmod.
Find the recipe providing /bin/kmod:
$ oe-pkgdata-util find-path /bin/kmod
kmod: /bin/kmod
Ok, could have guessed that it is kmod
.
Relevant line in the recipe: https://github.com/openembedded/openembedded-core/blob/5e7d09142da82c37aeab22c34d5314187c90bd84/meta/recipes-kernel/kmod/kmod_29.bb#L35
Fix
Enable XZ for kmod:
$ echo 'PACKAGECONFIG:append:pn-kmod = " xz"' >> meta-foo/conf/distro/foo-distro.conf
After building a new image XZ was enabled for kmod but the kernel modules still haven't been loaded:
root@raspberrypi4-64:~# kmod --version
kmod version 29
-ZSTD +XZ +ZLIB -LIBCRYPTO -EXPERIMENTAL
root@raspberrypi4-64:~# lsmod
Module Size Used by
However, after generating modules.dep and map files followed by a reboot the kernel modules where finally loaded.
root@raspberrypi4-64:~# depmod --all
root@raspberrypi4-64:~# reboot
root@raspberrypi4-64:~# lsmod
Module Size Used by
rfcomm 53248 2
cmac 16384 3
algif_hash 20480 1
aes_arm64 16384 3
algif_skcipher 20480 1
af_alg 32768 6 algif_hash,algif_skcipher
bnep 28672 2
hci_uart 49152 1
btbcm 28672 1 hci_uart
bluetooth 483328 31 hci_uart,btbcm,bnep,rfcomm
ecdh_generic 16384 2 bluetooth
ecc 36864 1 ecdh_generic
rpivid_mem 16384 0
brcmfmac 339968 0
bcm2835_codec 53248 0
bcm2835_v4l2 45056 0
bcm2835_isp 32768 0
brcmutil 24576 1 brcmfmac
v3d 86016 5
v4l2_mem2mem 45056 1 bcm2835_codec
bcm2835_mmal_vchiq 40960 3 bcm2835_codec,bcm2835_v4l2,bcm2835_isp
videobuf2_dma_contig 24576 2 bcm2835_codec,bcm2835_isp
gpu_sched 49152 1 v3d
videobuf2_vmalloc 20480 1 bcm2835_v4l2
videobuf2_memops 20480 2 videobuf2_vmalloc,videobuf2_dma_contig
videobuf2_v4l2 32768 4 bcm2835_codec,bcm2835_v4l2,v4l2_mem2mem,bcm2835_isp
raspberrypi_hwmon 16384 0
i2c_brcmstb 20480 0
videobuf2_common 69632 8 bcm2835_codec,videobuf2_vmalloc,videobuf2_dma_contig,videobuf2_v4l2,bcm2835_v4l2,v4l2_mem2mem,videobuf2_memops,bcm2835_isp
videodev 266240 6 bcm2835_codec,videobuf2_v4l2,bcm2835_v4l2,videobuf2_common,v4l2_mem2mem,bcm2835_isp
vc_sm_cma 36864 2 bcm2835_mmal_vchiq,bcm2835_isp
mc 61440 6 videodev,bcm2835_codec,videobuf2_v4l2,videobuf2_common,v4l2_mem2mem,bcm2835_isp
hid_lg_g15 20480 0
uio_pdrv_genirq 16384 0
cfg80211 876544 1 brcmfmac
uio 24576 1 uio_pdrv_genirq
nvmem_rmem 16384 0
rfkill 36864 4 bluetooth,cfg80211
sch_fq_codel 20480 6
fuse 139264 1
ipv6 561152 38
Hey @MatthiasGugel thanks for the explaination, I dig through it.
BTW this behaviour is only for 5.15 kernel and not present in 5.10. As we still have a 5.10 recipe here, so built it and everything was working fine. Forgot to update, timecrunch ;)
So need to check the 5.15 recipe or probably the upstream changes.
Though updates might be delayed as I only get time in weekends nowadays :)
@MatthiasGugel check this, I think this is the change which prevents loading of the kernel https://github.com/raspberrypi/linux/pull/4991
@MatthiasGugel @bhargavthriler It looks like kmod can't load the kernel modules due to the disabled compression type. xz
is not enabled by default in kmod
. That being said, give this MR a spin https://github.com/agherzan/meta-raspberrypi/pull/1056
@agherzan While #1056 enables xz
for kmod the kernel modules are still not loaded without depmod --all
:
root@raspberrypi4-64:~# kmod --version
kmod version 29
-ZSTD +XZ +ZLIB -LIBCRYPTO -EXPERIMENTAL
root@raspberrypi4-64:~# lsmod
Module Size Used by
root@raspberrypi4-64:~# depmod --all
root@raspberrypi4-64:~# reboot
root@raspberrypi4-64:~# lsmod
Module Size Used by
rfcomm 53248 2
cmac 16384 3
algif_hash 20480 1
aes_arm64 16384 3
algif_skcipher 20480 1
af_alg 32768 6 algif_hash,algif_skcipher
bnep 28672 2
hci_uart 49152 1
btbcm 28672 1 hci_uart
bluetooth 483328 31 hci_uart,btbcm,bnep,rfcomm
ecdh_generic 16384 2 bluetooth
ecc 36864 1 ecdh_generic
hid_lg_g15 20480 0
brcmfmac 339968 0
bcm2835_v4l2 45056 0
brcmutil 24576 1 brcmfmac
bcm2835_codec 53248 0
bcm2835_isp 32768 0
bcm2835_mmal_vchiq 40960 3 bcm2835_codec,bcm2835_v4l2,bcm2835_isp
v4l2_mem2mem 45056 1 bcm2835_codec
videobuf2_vmalloc 20480 1 bcm2835_v4l2
videobuf2_dma_contig 24576 2 bcm2835_codec,bcm2835_isp
v3d 86016 3
videobuf2_memops 20480 2 videobuf2_vmalloc,videobuf2_dma_contig
raspberrypi_hwmon 16384 0
videobuf2_v4l2 32768 4 bcm2835_codec,bcm2835_v4l2,v4l2_mem2mem,bcm2835_isp
gpu_sched 49152 1 v3d
videobuf2_common 69632 8 bcm2835_codec,videobuf2_vmalloc,videobuf2_dma_contig,videobuf2_v4l2,bcm2835_v4l2,v4l2_mem2mem,videobuf2_memops,bcm2835_isp
videodev 266240 6 bcm2835_codec,videobuf2_v4l2,bcm2835_v4l2,videobuf2_common,v4l2_mem2mem,bcm2835_isp
i2c_brcmstb 20480 0
cfg80211 876544 1 brcmfmac
vc_sm_cma 36864 2 bcm2835_mmal_vchiq,bcm2835_isp
mc 61440 6 videodev,bcm2835_codec,videobuf2_v4l2,videobuf2_common,v4l2_mem2mem,bcm2835_isp
rpivid_mem 16384 0
rfkill 36864 4 bluetooth,cfg80211
nvmem_rmem 16384 0
uio_pdrv_genirq 16384 0
uio 24576 1 uio_pdrv_genirq
sch_fq_codel 20480 6
fuse 139264 1
ipv6 561152 44
Any ideas on how to get the modules loaded without manual interaction? It worked previously.
That is unexpected - the build system should run depmod
at rootfs type. Did you build a clean image with the change above?
@agherzan ,yup @MatthiasGugel is correct even observing the same issue need to reboot after depmod --all
to load the modules.
I have also observed that modules.dep
is populated after your changes but somehow those are not loaded.
@bhargavthriler @MatthiasGugel please test https://lists.openembedded.org/g/openembedded-core/message/164932
delete your ${TMPDIR}
dir and build the rootfs+image again so that image post processing can re-run the depmod during image build.
@kraj , I deleted the tmpdir as you suggested and built the image again but still it's not loading all the drivers though those are present in module.deps
only these are loaded in the first boot
root@raspberrypi3:~# lsmod
Module Size Used by
rfcomm 49152 2
cmac 16384 3
algif_hash 16384 1
nfc 86016 0
aes_arm_bs 24576 2
crypto_simd 16384 1 aes_arm_bs
cryptd 24576 2 crypto_simd
algif_skcipher 16384 1
af_alg 28672 6 algif_hash,algif_skcipher
bnep 20480 2
hci_uart 40960 1
btbcm 20480 1 hci_uart
bluetooth 421888 31 hci_uart,bnep,btbcm,rfcomm
ecdh_generic 16384 2 bluetooth
ecc 40960 1 ecdh_generic
rfkill 32768 3 bluetooth,nfc
ipv6 520192 26
The following are loaded after executing depmod -a
root@raspberrypi3:~# depmod -a
root@raspberrypi3:~# reboot
Broadcast message from root@raspberrypi3 (ttyS0) (Fri Mar 9 12:40:58 2018):
The system is going down for reboot NOW!
INIT: Sending processes configured via /etc/inittab the TERM signal
Stopping OpenBSD Secure Shell server: sshdstopped /usr/sbin/sshd (pid 421)
.
* Stopping Avahi mDNS/DNS-SD Daemon: avahi-daemon
...done.
Stopping advanced power management daemon: no /usr/sbin/apmd found; none killed
apmd.
----------
reboot
----------
raspber[ 20.160226] Bluetooth: RFCOMM ver 1.11
rypi3 login: [ 33.770185] cam-dummy-reg: disabling
root
root@raspberrypi3:~# lsmod
Module Size Used by
rfcomm 49152 2
cmac 16384 3
algif_hash 16384 1
nfc 86016 0
aes_arm_bs 24576 2
crypto_simd 16384 1 aes_arm_bs
cryptd 24576 2 crypto_simd
algif_skcipher 16384 1
af_alg 28672 6 algif_hash,algif_skcipher
bnep 20480 2
hci_uart 40960 1
btbcm 20480 1 hci_uart
bluetooth 421888 31 hci_uart,bnep,btbcm,rfcomm
ecdh_generic 16384 2 bluetooth
ecc 40960 1 ecdh_generic
ipv6 520192 26
brcmfmac 335872 0
brcmutil 24576 1 brcmfmac
bcm2835_codec 45056 0
bcm2835_isp 32768 0
sha256_generic 16384 0
v4l2_mem2mem 36864 1 bcm2835_codec
bcm2835_v4l2 45056 0
bcm2835_mmal_vchiq 32768 3 bcm2835_isp,bcm2835_codec,bcm2835_v4l2
cfg80211 770048 1 brcmfmac
videobuf2_dma_contig 20480 2 bcm2835_isp,bcm2835_codec
videobuf2_vmalloc 16384 1 bcm2835_v4l2
videobuf2_memops 16384 2 videobuf2_dma_contig,videobuf2_vmalloc
videobuf2_v4l2 32768 4 bcm2835_isp,bcm2835_codec,bcm2835_v4l2,v4l2_mem2mem
videobuf2_common 65536 8 bcm2835_isp,bcm2835_codec,videobuf2_dma_contig,videobuf2_vmalloc,videobuf2_memops,bcm2835_v4l2,v4l2_mem2mem,videobuf2_v4l2
videodev 245760 6 bcm2835_isp,bcm2835_codec,videobuf2_common,bcm2835_v4l2,v4l2_mem2mem,videobuf2_v4l2
rfkill 32768 4 bluetooth,nfc,cfg80211
mc 45056 6 bcm2835_isp,bcm2835_codec,videobuf2_common,videodev,v4l2_mem2mem,videobuf2_v4l2
vc_sm_cma 32768 2 bcm2835_isp,bcm2835_mmal_vchiq
raspberrypi_hwmon 16384 0
uio_pdrv_genirq 16384 0
uio 20480 1 uio_pdrv_genirq
fixed 16384 0
root@raspberrypi3:~#
interesting, I have not done raspberrypi3
builds but on rpi4 it looks ok. What do you use for a build host ? I wonder if thats the variable factor.
I am using Ubuntu 20.04. Build image: core-image-base
on pi4 the order of modules is a bit different after depmod -a
but content is same
--- a
+++ b
@@ -1,30 +1,28 @@
-root@raspberrypi4-64:~# lsmod
-Module Size Used by
-v3d 86016 1
-bcm2835_v4l2 45056 0
-bcm2835_isp 32768 0
bcm2835_codec 53248 0
+bcm2835_isp 32768 0
+v3d 86016 1
brcmfmac 339968 0
-gpu_sched 49152 1 v3d
-v4l2_mem2mem 45056 1 bcm2835_codec
+bcm2835_v4l2 45056 0
bcm2835_mmal_vchiq 40960 3 bcm2835_codec,bcm2835_v4l2,bcm2835_isp
-brcmutil 24576 1 brcmfmac
-raspberrypi_hwmon 16384 0
videobuf2_vmalloc 20480 1 bcm2835_v4l2
+v4l2_mem2mem 45056 1 bcm2835_codec
videobuf2_dma_contig 24576 2 bcm2835_codec,bcm2835_isp
videobuf2_memops 20480 2 videobuf2_vmalloc,videobuf2_dma_contig
videobuf2_v4l2 32768 4 bcm2835_codec,bcm2835_v4l2,v4l2_mem2mem,bcm2835_isp
videobuf2_common 69632 8 bcm2835_codec,videobuf2_vmalloc,videobuf2_dma_contig,videobuf2_v4l2,bcm2835_v4l2,v4l2_mem2mem,videobuf2_memops,bcm2835_isp
-i2c_brcmstb 20480 0
+gpu_sched 49152 1 v3d
+brcmutil 24576 1 brcmfmac
videodev 266240 6 bcm2835_codec,videobuf2_v4l2,bcm2835_v4l2,videobuf2_common,v4l2_mem2mem,bcm2835_isp
+raspberrypi_hwmon 16384 0
+i2c_brcmstb 20480 0
vc_sm_cma 36864 2 bcm2835_mmal_vchiq,bcm2835_isp
mc 61440 6 videodev,bcm2835_codec,videobuf2_v4l2,videobuf2_common,v4l2_mem2mem,bcm2835_isp
-cfg80211 876544 1 brcmfmac
rpivid_mem 16384 0
-rfkill 36864 2 cfg80211
-nvmem_rmem 16384 0
+cfg80211 876544 1 brcmfmac
uio_pdrv_genirq 16384 0
uio 24576 1 uio_pdrv_genirq
+nvmem_rmem 16384 0
+rfkill 36864 2 cfg80211
sch_fq_codel 20480 7
fuse 139264 1
ipv6 565248 32
I am using Ubuntu 20.04. Build image: core-image-base
OK I dont ubuntu instead use archlinux but are you wiping the SD card clean ? or is there some persistence there.
I am using Ubuntu 20.04. Build image: core-image-base
OK I dont ubuntu instead use archlinux but are you wiping the SD card clean ? or is there some persistence there.
Shouldn't be any, i am using bmaptool to flash.
Ok tonight i will trigger a scratch build.
Btw are the changes in oe-core merged? Currently i am using @agherzan 's fork .
For me the kernel modules are now loaded without the need to call depmod --all
manually. My host is Ubuntu 20.04 but I am using kas to build the image inside a docker container.
configuration.yml:
header:
version: 11
machine: raspberrypi4-64
distro: my-distro
target: my-image
build_system: oe
repos:
openembedded-core:
url: "https://git.openembedded.org/openembedded-core"
refspec: "master"
path: "layers/openembedded-core"
layers:
meta:
...
meta-raspberrypi:
url: "https://github.com/agherzan/meta-raspberrypi"
refspec: "master"
path: "layers/meta-raspberrypi"
...
I manually patched kas/layers/openembedded-core/meta/recipes-kernel/kmod/kmod_29.bb
and deleted what I believe is ${TMPDIR}
bevor building the image:
$ rm -rf kas/build/tmp-glibc
$ kas/kas-container build kas/configuration.yml
...
WARNING - Repo openembedded-core is dirty - no checkout
...
0: my-image-1.0-r0 do_rootfs - 5s (pid 375424) 3% |#######
Flash the SD Card:
$ sudo umount /dev/mmcblk0p*; sudo bmaptool copy my-image-raspberrypi4-64.wic.bz2 /dev/mmcblk0"
root@raspberrypi4-64:~# lsmod
Module Size Used by
rfcomm 53248 2
cmac 16384 3
algif_hash 20480 1
aes_arm64 16384 3
algif_skcipher 20480 1
af_alg 32768 6 algif_hash,algif_skcipher
bnep 28672 2
hci_uart 49152 1
btbcm 28672 1 hci_uart
bluetooth 483328 31 hci_uart,btbcm,bnep,rfcomm
ecdh_generic 16384 2 bluetooth
ecc 36864 1 ecdh_generic
rpivid_mem 16384 0
brcmfmac 339968 0
brcmutil 24576 1 brcmfmac
hid_lg_g15 20480 0
bcm2835_v4l2 45056 0
bcm2835_codec 53248 0
v4l2_mem2mem 45056 1 bcm2835_codec
bcm2835_isp 32768 0
bcm2835_mmal_vchiq 40960 3 bcm2835_codec,bcm2835_v4l2,bcm2835_isp
videobuf2_vmalloc 20480 1 bcm2835_v4l2
videobuf2_dma_contig 24576 2 bcm2835_codec,bcm2835_isp
videobuf2_memops 20480 2 videobuf2_vmalloc,videobuf2_dma_contig
videobuf2_v4l2 32768 4 bcm2835_codec,bcm2835_v4l2,v4l2_mem2mem,bcm2835_isp
v3d 86016 3
videobuf2_common 69632 8 bcm2835_codec,videobuf2_vmalloc,videobuf2_dma_contig,videobuf2_v4l2,bcm2835_v4l2,v4l2_mem2mem,videobuf2_memops,bcm2835_isp
gpu_sched 49152 1 v3d
videodev 266240 6 bcm2835_codec,videobuf2_v4l2,bcm2835_v4l2,videobuf2_common,v4l2_mem2mem,bcm2835_isp
raspberrypi_hwmon 16384 0
mc 61440 6 videodev,bcm2835_codec,videobuf2_v4l2,videobuf2_common,v4l2_mem2mem,bcm2835_isp
vc_sm_cma 36864 2 bcm2835_mmal_vchiq,bcm2835_isp
i2c_brcmstb 20480 0
nvmem_rmem 16384 0
uio_pdrv_genirq 16384 0
uio 24576 1 uio_pdrv_genirq
cfg80211 876544 1 brcmfmac
rfkill 36864 4 bluetooth,cfg80211
sch_fq_codel 20480 6
fuse 139264 1
ipv6 561152 44
So for me https://lists.openembedded.org/g/openembedded-core/message/164932 and deleting ${TMPDIR}
resolved all my issues.
@MatthiasGugel Thanks for confirming. @kraj We need that in kirkstone too - that as in the oe-core change.
The MR automatically closed this issue - and it is fine as we fixed it from this layer's perspective, but we still need to deal with it being broken in kirkstone. It will be backported/cherry-pick in the next round.
${TMPDIR}
before build did not fix the issue with raspberrypi0-wifi
. depmod -a
followed by reboot works. I have temporarily patched kernel build configuration so CONFIG_MODULE_COMPRESS_XZ=n
and everything works as before.@tewarid That is expected. You need the oe-core patch mentioned above.
@tewarid That is expected. You need the oe-core patch mentioned above.
Thanks - sounds good. I'll wait for the patch to make it to the repo.
Description Observed that some kernel modules are not being loaded in the latest kernel
5.15.34-v7
.So I have built a
core-image-base
with the tip of the master (0135a02
) and while trying access the camera using Picamera got some errors. The errors mainly complain about mmal drivers not present.After digging through my system found an older build (don't know why didn't delete it but thankfully it gave some insight into the issue), I tried booting that image and everything seems to working fine.
So I checked out to the commit which the older build was using (
63a3d8cb17c5d1affe8f2848f45fcc6a706f9412
), and the camera worked fine(though I had to make few changes, which are not significant for this issue). While analyzing the bootlogs found that the latest build (0135a02
) doesn't load all the drivers.Also I have observed that the kernel module are compressed in the 5.15.34 kernel, eg:
root@raspberrypi3:~# ls /lib/modules/5.15.34-v7/kernel/drivers/usb/gadget/libcomposite.ko.xz
and while trying load the modules using modprobe getting the following error:My question is what and where the changes have happened to the kernel between
63a3d8cb17c5d1affe8f2848f45fcc6a706f9412
(5.10) and0135a02
(5.15) , so that I can look into and adapt the changes required ?Note: All the commit which are mentioned above are of this repo.
Logs I have attached the bootlogs of both the kernels. lsmod logs
Additional details (revisions used, host distro, etc.):
Some more details of the builds
64156a6f8f3b4527c4732c91936aee0a167acb48
5ec6839709f1f9411a8334306c563675246fff35
IMAGE_INSTALL:append = "python3-picamera openssh libcamera libffi libgphoto2"
BBLAYERS ?= " \ /home/bulbasaur/HDD-2-2/RPI-camera-project-23042022/poky/meta \ /home/bulbasaur/HDD-2-2/RPI-camera-project-23042022/poky/meta-poky \ /home/bulbasaur/HDD-2-2/RPI-camera-project-23042022/poky/meta-yocto-bsp \ /home/bulbasaur/HDD-2-2/RPI-camera-project-23042022/meta-raspberrypi \ /home/bulbasaur/HDD-2-2/RPI-camera-project-23042022/meta-openembedded/meta-python \ /home/bulbasaur/HDD-2-2/RPI-camera-project-23042022/meta-openembedded/meta-oe \ /home/bulbasaur/HDD-2-2/RPI-camera-project-23042022/meta-openembedded/meta-multimedia \ "