search

jens-maus / RaspberryMatic

:house: A feature-rich but lightweight, buildroot-based Linux operating system alternative for your CloudFree CCU3/ELV-Charly 'homematicIP CCU' IoT smarthome central. Running as a pure virtual appliance (ProxmoxVE, Home Assistant, LXC, Docker/OCI, Kubernetes/K8s, etc.) or on a dedicated embedded device (RaspberryPi, Tinkerboard, IntelNUC, etc.)
https://raspberrymatic.de
Apache License 2.0
1.5k stars 184 forks source link

piVCCU kernel module not loading on Ubuntu 24.04 #2726

Closed gudnithor closed 2 months ago

gudnithor commented 2 months ago

Describe the issue you are experiencing

Hi - following the guide here: https://github.com/jens-maus/RaspberryMatic/wiki/Installation-Docker-OCI

Doing the manual install and when running sudo apt install pivccu-modules-dkms the modules will not be loaded. The error given is:

Preparing to unpack .../pivccu-modules-dkms_1.0.84_all.deb ...
Unpacking pivccu-modules-dkms (1.0.84) ...
Setting up pivccu-modules-dkms (1.0.84) ...
Create kernel modules ... FAILED
Check kernel headers ... Done
Prepare kernel headers ... Done
Install DKMS package ... Done
Try to load fresh build modules ... FAILED
modprobe: ERROR: could not insert 'generic_raw_uart': Exec format error
Enable DKMS service ... Done

From what I understand the Exec format error has something to do with a mismatch between the kernel headers used and the currently running kernel. This is on a fresh install of Ubuntu.

This error comes in dmesg: [ 334.714632] module generic_raw_uart: .gnu.linkonce.this_module section size must match the kernel's built struct module size at run time

I suspect that this should probably be filed under the piVCCU project but I think Alex is adamant that he only supports arm.

Describe the behavior you expected

piVCCU kernel modules loaded successfully.

Steps to reproduce the issue

  1. Setup a fresh install of Ubuntu Server 24.04 on a x86_64 machine.
  2. Follow the manual setup instructions here
  3. Module fails to load correctly

What is the version this bug report is based on?

N/A

Which base platform are you running?

intelnuc (intel NUC, AMD64/x86_64)

Which HomeMatic/homematicIP radio module are you using?

HmIP-RFUSB

Anything in the logs that might be useful for us?

There is nothing more in any logs that I think is relevant. There are no errors in dkms when building the modules.

Additional information

No response

gudnithor commented 2 months ago 
gudnithor@nuc:~$ uname -a
Linux nuc 6.8.0-31-generic #31-Ubuntu SMP PREEMPT_DYNAMIC Sat Apr 20 00:40:06 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
gudnithor@nuc:~$ cat /proc/version
Linux version 6.8.0-31-generic (buildd@lcy02-amd64-080) (x86_64-linux-gnu-gcc-13 (Ubuntu 13.2.0-23ubuntu4) 13.2.0, GNU ld (GNU Binutils for Ubuntu) 2.42) #31-Ubuntu SMP PREEMPT_DYNAMIC Sat Apr 20 00:40:06 UTC 2024
gudnithor@nuc:~$ ls -l /var/lib/dkms/pivccu/
total 4
drwxr-xr-x 3 root root 4096 Apr 28 20:50 1.0.84
lrwxrwxrwx 1 root root   30 Apr 28 20:50 kernel-6.8.0-31-generic-x86_64 -> 1.0.84/6.8.0-31-generic/x86_64
gudnithor@nuc:~$ apt info binutils
Package: binutils
Version: 2.42-4ubuntu2
gudnithor@nuc:~$ apt list | grep header | grep installed

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

linux-headers-6.8.0-31-generic/noble,now 6.8.0-31.31 amd64 [installed,automatic]
linux-headers-6.8.0-31/noble,now 6.8.0-31.31 all [installed,automatic]
linux-headers-generic/noble,now 6.8.0-31.31 amd64 [installed,automatic]
gudnithor@nuc:~$ apt info gcc
Package: gcc
Version: 4:13.2.0-7ubuntu1

I see no mismatch in the compiler, headers, etc which should be causing this.

The make log from dkms:

gudnithor@nuc:/var/lib/dkms/pivccu/kernel-6.8.0-31-generic-x86_64/log$ cat make.log
DKMS make.log for pivccu-1.0.84 for kernel 6.8.0-31-generic (x86_64)
Sun Apr 28 20:50:55 UTC 2024
make -C /lib/modules/6.8.0-31-generic/build M=/var/lib/dkms/pivccu/1.0.84/build modules
make[1]: Entering directory '/usr/src/linux-headers-6.8.0-31-generic'
  CC [M]  /var/lib/dkms/pivccu/1.0.84/build/eq3_char_loop.o
  CC [M]  /var/lib/dkms/pivccu/1.0.84/build/plat_eq3ccu2.o
  CC [M]  /var/lib/dkms/pivccu/1.0.84/build/generic_raw_uart.o
  CC [M]  /var/lib/dkms/pivccu/1.0.84/build/pl011_raw_uart.o
  CC [M]  /var/lib/dkms/pivccu/1.0.84/build/dw_apb_raw_uart.o
  CC [M]  /var/lib/dkms/pivccu/1.0.84/build/meson_raw_uart.o
  CC [M]  /var/lib/dkms/pivccu/1.0.84/build/fake_hmrf.o
  CC [M]  /var/lib/dkms/pivccu/1.0.84/build/rpi_rf_mod_led.o
  CC [M]  /var/lib/dkms/pivccu/1.0.84/build/dummy_rx8130.o
  CC [M]  /var/lib/dkms/pivccu/1.0.84/build/led_trigger_timer.o
  CC [M]  /var/lib/dkms/pivccu/1.0.84/build/hb_rf_usb.o
  CC [M]  /var/lib/dkms/pivccu/1.0.84/build/hb_rf_usb_2.o
  CC [M]  /var/lib/dkms/pivccu/1.0.84/build/hb_rf_eth.o
/var/lib/dkms/pivccu/1.0.84/build/generic_raw_uart.c:824:5: warning: no previous prototype for ‘generic_raw_uart_get_gpio_index’ [-Wmissing-prototypes]
  824 | int generic_raw_uart_get_gpio_index(struct device *dev, char *con_id, unsigned int idx)
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/var/lib/dkms/pivccu/1.0.84/build/generic_raw_uart.c:839:5: warning: no previous prototype for ‘generic_raw_uart_get_led_gpio_index’ [-Wmissing-prototypes]
  839 | int generic_raw_uart_get_led_gpio_index(struct generic_raw_uart_instance *instance, struct device *dev, enum generic_raw_uart_led led)
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/var/lib/dkms/pivccu/1.0.84/build/generic_raw_uart.c:971:5: warning: no previous prototype for ‘generic_raw_uart_probe_rtc_device’ [-Wmissing-prototypes]
  971 | int generic_raw_uart_probe_rtc_device(struct device *dev, bool *rtc_detected)
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  CC [M]  /var/lib/dkms/pivccu/1.0.84/build/rtc-rx8130.o
  MODPOST /var/lib/dkms/pivccu/1.0.84/build/Module.symvers
  CC [M]  /var/lib/dkms/pivccu/1.0.84/build/eq3_char_loop.mod.o
  CC [M]  /var/lib/dkms/pivccu/1.0.84/build/plat_eq3ccu2.mod.o
  CC [M]  /var/lib/dkms/pivccu/1.0.84/build/generic_raw_uart.mod.o
  CC [M]  /var/lib/dkms/pivccu/1.0.84/build/pl011_raw_uart.mod.o
  CC [M]  /var/lib/dkms/pivccu/1.0.84/build/dw_apb_raw_uart.mod.o
  CC [M]  /var/lib/dkms/pivccu/1.0.84/build/meson_raw_uart.mod.o
  CC [M]  /var/lib/dkms/pivccu/1.0.84/build/fake_hmrf.mod.o
  CC [M]  /var/lib/dkms/pivccu/1.0.84/build/rpi_rf_mod_led.mod.o
  CC [M]  /var/lib/dkms/pivccu/1.0.84/build/dummy_rx8130.mod.o
  CC [M]  /var/lib/dkms/pivccu/1.0.84/build/led_trigger_timer.mod.o
  CC [M]  /var/lib/dkms/pivccu/1.0.84/build/hb_rf_usb.mod.o
  CC [M]  /var/lib/dkms/pivccu/1.0.84/build/hb_rf_usb_2.mod.o
  CC [M]  /var/lib/dkms/pivccu/1.0.84/build/hb_rf_eth.mod.o
  CC [M]  /var/lib/dkms/pivccu/1.0.84/build/rtc-rx8130.mod.o
  LD [M]  /var/lib/dkms/pivccu/1.0.84/build/eq3_char_loop.ko
  LD [M]  /var/lib/dkms/pivccu/1.0.84/build/plat_eq3ccu2.ko
  LD [M]  /var/lib/dkms/pivccu/1.0.84/build/generic_raw_uart.ko
  LD [M]  /var/lib/dkms/pivccu/1.0.84/build/pl011_raw_uart.ko
  LD [M]  /var/lib/dkms/pivccu/1.0.84/build/dw_apb_raw_uart.ko
  LD [M]  /var/lib/dkms/pivccu/1.0.84/build/meson_raw_uart.ko
  LD [M]  /var/lib/dkms/pivccu/1.0.84/build/fake_hmrf.ko
  LD [M]  /var/lib/dkms/pivccu/1.0.84/build/rpi_rf_mod_led.ko
  LD [M]  /var/lib/dkms/pivccu/1.0.84/build/led_trigger_timer.ko
  LD [M]  /var/lib/dkms/pivccu/1.0.84/build/rtc-rx8130.ko
  LD [M]  /var/lib/dkms/pivccu/1.0.84/build/dummy_rx8130.ko
  LD [M]  /var/lib/dkms/pivccu/1.0.84/build/hb_rf_usb_2.ko
  LD [M]  /var/lib/dkms/pivccu/1.0.84/build/hb_rf_usb.ko
  LD [M]  /var/lib/dkms/pivccu/1.0.84/build/hb_rf_eth.ko
make[1]: Leaving directory '/usr/src/linux-headers-6.8.0-31-generic'
jens-maus commented 2 months ago

Have you tried to use the install-docker.sh installation script as explained in the documentation? This might/could help in case some dependencies are actually missing with your manual install approach.

In addition, please see the following section in the documentation explaining potential solutions for these kind of Exec format error issues: https://github.com/jens-maus/RaspberryMatic/wiki/Installation-Docker-OCI#potential-container-startup-errors

gudnithor commented 2 months ago

Have you tried to use the install-docker.sh installation script as explained in the documentation? This might/could help in case some dependencies are actually missing with your manual install approach.

No, I didn't try using that script - I did however just right now read through it and I don't spot anything that script is doing that I didn't try before manually; especially regards to some missing or wrong dependancies.

In addition, please see the following section in the documentation explaining potential solutions for these kind of Exec format error issues: https://github.com/jens-maus/RaspberryMatic/wiki/Installation-Docker-OCI#potential-container-startup-errors

Tried this - same result as before. See log:

gudnithor@nuc:~$ uname -a
Linux nuc 6.8.0-31-generic #31-Ubuntu SMP PREEMPT_DYNAMIC Sat Apr 20 00:40:06 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
gudnithor@nuc:~$ sudo apt update
[sudo] password for gudnithor:
Hit:4 http://security.ubuntu.com/ubuntu noble-security InRelease
Hit:1 https://is.archive.ubuntu.com/ubuntu noble InRelease
Get:2 https://is.archive.ubuntu.com/ubuntu noble-updates InRelease [89.7 kB]
Get:3 https://is.archive.ubuntu.com/ubuntu noble-backports InRelease [90.8 kB]
Hit:5 https://apt.pivccu.de/piVCCU stable InRelease
Fetched 180 kB in 0s (391 kB/s)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
All packages are up to date.
gudnithor@nuc:~$ sudo apt remove --purge linux-headers-*
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done

[...lots of "Note, selecting 'linux-headers-generic-hwe-24.04' for glob 'linux-headers-*'" lines removed for readability...]

The following packages were automatically installed and are no longer required:
  bpfcc-tools bpftrace ieee-data libbpfcc libclang-cpp18 libclang1-18
  libllvm18 python3-bpfcc python3-netaddr ubuntu-kernel-accessories
Use 'sudo apt autoremove' to remove them.
The following packages will be REMOVED:
  linux-generic* linux-headers-6.8.0-31* linux-headers-6.8.0-31-generic*
  linux-headers-generic*
0 upgraded, 0 newly installed, 4 to remove and 0 not upgraded.
After this operation, 115 MB disk space will be freed.
Do you want to continue? [Y/n] y
(Reading database ... 86089 files and directories currently installed.)
Removing linux-generic (6.8.0-31.31) ...
Removing linux-headers-generic (6.8.0-31.31) ...
Removing linux-headers-6.8.0-31-generic (6.8.0-31.31) ...
dpkg: warning: while removing linux-headers-6.8.0-31-generic, directory '/usr/src/linux-headers-6.8.0-31-generic/scripts/kconfig' not empty so not removed
dpkg: warning: while removing linux-headers-6.8.0-31-generic, directory '/usr/src/linux-headers-6.8.0-31-generic/scripts/genksyms' not empty so not removed
dpkg: warning: while removing linux-headers-6.8.0-31-generic, directory '/usr/src/linux-headers-6.8.0-31-generic/scripts/basic' not empty so not removed
dpkg: warning: while removing linux-headers-6.8.0-31-generic, directory '/usr/src/linux-headers-6.8.0-31-generic/include/config' not empty so not removed
Removing linux-headers-6.8.0-31 (6.8.0-31.31) ...
gudnithor@nuc:~$ sudo rm -rf /usr/src/linux-headers-6.8.0-31-generic/
gudnithor@nuc:~$ sudo apt autoremove && sudo apt autoclean
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages will be REMOVED:
  bpfcc-tools bpftrace ieee-data libbpfcc libclang-cpp18 libclang1-18
  libllvm18 python3-bpfcc python3-netaddr ubuntu-kernel-accessories
0 upgraded, 0 newly installed, 10 to remove and 0 not upgraded.
After this operation, 246 MB disk space will be freed.
Do you want to continue? [Y/n] y
(Reading database ... 55830 files and directories currently installed.)
Removing bpfcc-tools (0.29.1+ds-1ubuntu7) ...
Removing bpftrace (0.20.2-1ubuntu4) ...
Removing python3-netaddr (0.8.0-2ubuntu1) ...
Removing ieee-data (20220827.1) ...
Removing python3-bpfcc (0.29.1+ds-1ubuntu7) ...
Removing libbpfcc:amd64 (0.29.1+ds-1ubuntu7) ...
Removing libclang-cpp18 (1:18.1.3-1) ...
Removing libclang1-18 (1:18.1.3-1) ...
Removing libllvm18:amd64 (1:18.1.3-1) ...
Removing ubuntu-kernel-accessories (1.539) ...
Processing triggers for man-db (2.12.0-4build2) ...
Processing triggers for libc-bin (2.39-0ubuntu8.1) ...
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
gudnithor@nuc:~$ sudo apt install linux-headers-generic
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  linux-headers-6.8.0-31 linux-headers-6.8.0-31-generic
The following NEW packages will be installed:
  linux-headers-6.8.0-31 linux-headers-6.8.0-31-generic linux-headers-generic
0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
Need to get 17.5 MB of archives.
After this operation, 114 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 https://is.archive.ubuntu.com/ubuntu noble/main amd64 linux-headers-6.8.0-31 all 6.8.0-31.31 [13.6 MB]
Get:2 https://is.archive.ubuntu.com/ubuntu noble/main amd64 linux-headers-6.8.0-31-generic amd64 6.8.0-31.31 [3866 kB]
Get:3 https://is.archive.ubuntu.com/ubuntu noble/main amd64 linux-headers-generic amd64 6.8.0-31.31 [9834 B]
Fetched 17.5 MB in 2s (8199 kB/s)
Selecting previously unselected package linux-headers-6.8.0-31.
(Reading database ... 55053 files and directories currently installed.)
Preparing to unpack .../linux-headers-6.8.0-31_6.8.0-31.31_all.deb ...
Unpacking linux-headers-6.8.0-31 (6.8.0-31.31) ...
Selecting previously unselected package linux-headers-6.8.0-31-generic.
Preparing to unpack .../linux-headers-6.8.0-31-generic_6.8.0-31.31_amd64.deb ...
Unpacking linux-headers-6.8.0-31-generic (6.8.0-31.31) ...
Selecting previously unselected package linux-headers-generic.
Preparing to unpack .../linux-headers-generic_6.8.0-31.31_amd64.deb ...
Unpacking linux-headers-generic (6.8.0-31.31) ...
Setting up linux-headers-6.8.0-31 (6.8.0-31.31) ...
Setting up linux-headers-6.8.0-31-generic (6.8.0-31.31) ...
/etc/kernel/header_postinst.d/dkms:
 * dkms: running auto installation service for kernel 6.8.0-31-generic
 * dkms: autoinstall for kernel 6.8.0-31-generic
   ...done.
Setting up linux-headers-generic (6.8.0-31.31) ...
Scanning processes...
Scanning processor microcode...
Scanning linux images...

Running kernel seems to be up-to-date.

The processor microcode seems to be up-to-date.

No services need to be restarted.

No containers need to be restarted.

No user sessions are running outdated binaries.

No VM guests are running outdated hypervisor (qemu) binaries on this host.
gudnithor@nuc:~$ sudo apt install --reinstall pivccu-modules-dkms
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
0 upgraded, 0 newly installed, 1 reinstalled, 0 to remove and 0 not upgraded.
Need to get 43.1 kB of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 https://apt.pivccu.de/piVCCU stable/main amd64 pivccu-modules-dkms all 1.0.84 [43.1 kB]
Fetched 43.1 kB in 0s (128 kB/s)
Preconfiguring packages ...
(Reading database ... 85309 files and directories currently installed.)
Preparing to unpack .../pivccu-modules-dkms_1.0.84_all.deb ...
Disabled DKMS service ... Done
Remove obsolete kernel modules ... Done
Unpacking pivccu-modules-dkms (1.0.84) over (1.0.84) ...
Setting up pivccu-modules-dkms (1.0.84) ...
Create kernel modules ... FAILED
Check kernel headers ... Done
Prepare kernel headers ... Done
Install DKMS package ... Done
Try to load fresh build modules ... FAILED
modprobe: ERROR: could not insert 'generic_raw_uart': Exec format error
Enable DKMS service ... Done
Scanning processes...
Scanning processor microcode...
Scanning linux images...

Running kernel seems to be up-to-date.

The processor microcode seems to be up-to-date.

No services need to be restarted.

No containers need to be restarted.

No user sessions are running outdated binaries.

No VM guests are running outdated hypervisor (qemu) binaries on this host.

Thank you Jens for the help!

kosekmi commented 2 months ago

I am having the exact same issue, happy to help if additional analyses or testing is required.

jens-maus commented 2 months ago

Actually, I just tested this on a vanilla Ubuntu 24.04 installation and my mentioned procedure (see https://github.com/jens-maus/RaspberryMatic/wiki/Installation-Docker-OCI#potential-container-startup-errors) worked out-of-the-box to get the kernel modules correctly compiled and modprobed.

See, here on a vanilla Ubuntu 24.04 system I could reproduce the issue that the initial instal-docker.sh execution fails with

[...]
Create kernel modules ... FAILED
Check kernel headers ... Done
Prepare kernel headers ... Done
Install DKMS package ... Done
Try to load fresh build modules ... FAILED
modprobe: ERROR: could not insert 'generic_raw_uart': Exec format error
[...]

Thus, the typical Exec format error issue. Nevertheless after executing the following commands:

sudo apt remove --purge linux-headers-*
sudo rm -rf /usr/src/linux-header-*
sudo apt autoremove && sudo apt autoclean
sudo apt install linux-headers-generic
sudo apt install --reinstall pivccu-modules-dkms

the kernel modules were correctly compiled in the last reinstall step via dkms even on Ubuntu 24.04:

[...]
administrator@ubuntu2404:~$ sudo apt install --reinstall pivccu-modules-dkms
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
0 upgraded, 0 newly installed, 1 reinstalled, 0 to remove and 0 not upgraded.
Need to get 43.1 kB of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 https://apt.pivccu.de/piVCCU stable/main amd64 pivccu-modules-dkms all 1.0.84 [43.1 kB]
Fetched 43.1 kB in 0s (203 kB/s)               
debconf: delaying package configuration, since apt-utils is not installed
(Reading database ... 75839 files and directories currently installed.)
Preparing to unpack .../pivccu-modules-dkms_1.0.84_all.deb ...
Disabled DKMS service ... Done
Remove obsolete kernel modules ... Done
Unpacking pivccu-modules-dkms (1.0.84) over (1.0.84) ...
debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 79.)
debconf: falling back to frontend: Readline
Setting up pivccu-modules-dkms (1.0.84) ...
debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 79.)
debconf: falling back to frontend: Readline
Create kernel modules ... Done
Enable DKMS service ... Done
Configuring pivccu-modules-dkms
-------------------------------

WARNING: A reboot is required to enable the modules.

Scanning processes...                                                                                     
Scanning linux images...                                                                                  

Running kernel seems to be up-to-date.

No services need to be restarted.

No containers need to be restarted.

No user sessions are running outdated binaries.

No VM guests are running outdated hypervisor (qemu) binaries on this host.

Note the Create kernel modules ... Done line and that no Exec format error is generated anymore.

So following the official documentation (see point 1 in https://github.com/jens-maus/RaspberryMatic/wiki/Installation-Docker-OCI#potential-container-startup-errors) did actually work in my case. In the end, this Exec format error clearly points at some incorrect differences between the currently running kernel and the installed kernel headers as it seems.

kosekmi commented 2 months ago

Thanks much, I forgot the restart following the installation of linux-headers-generic 🙄

jens-maus commented 2 months ago

As this is rather a point of discussion and not an issue in RaspberryMatic I will move it over to the discussions fora part for future reference.