tvheadend from Cloudsmith repository is not ARMv6-compatible

shawly commented 8 months ago

Creating a bug report/issue

[x] I have searched the existing open and closed issues

Required Information

DietPi version | cat /boot/dietpi/.version

G_DIETPI_VERSION_CORE=9
G_DIETPI_VERSION_SUB=1
G_DIETPI_VERSION_RC=1
G_GITBRANCH='master'
G_GITOWNER='MichaIng'
G_LIVE_PATCH_STATUS[0]='not applicable'

Distro version | echo $G_DISTRO_NAME $G_RASPBIAN

bookworm 1

Kernel version | uname -a

Linux vdr 6.1.21+ #1642 Mon Apr  3 17:19:14 BST 2023 armv6l GNU/Linux

SBC model | echo $G_HW_MODEL_NAME or (EG: RPi3)

RPi B (armv6l)

Power supply used | (EG: 5V 1A RAVpower)

5V 2.4A ANKER Powersupply

SD card used | (EG: SanDisk ultra)

SanDisk Extreme 8GB Class 10 UHS-I 30MB/Sec

Additional Information (if applicable)

Software title | (EG: Nextcloud)

docker and tvheadend

Was the software title installed freshly or updated/migrated?

Fresh install

Can this issue be replicated on a fresh installation of DietPi?

Yes

Bug report ID | echo $G_HW_UUID

d60148e6-c997-45ed-9110-7ed3044fb1e7

Steps to reproduce

For docker:

Install docker through dietpi-software
systemctl status docker
dockerd[2235]: time="2024-03-06T10:02:16.985388850+01:00" level=warning msg="Failed to run [runc features]: \"\"" error="signal: segmentation fault (core dumped)"

For tvheadend:

Install tvheadend repo through install script from https://tvheadend.org/p/downloads
apt install tvheadend
tvheadend.service won't start
Executing tvheadend results in "Illegal instructions"

Alertnatively installing another armhf build directly through any deb from https://cloudsmith.io/~tvheadend/repos/tvheadend/packages/?q=distribution%3Araspbian+AND+distribution%3Abookworm results in the same behavior

Expected behaviour

Working docker and tvheadend

Actual behaviour

Not working docker and tvheadend

Extra details

...

Joulinar commented 8 months ago

As far as I can see, you are using 32bit system (RPi B). However the tvheadend Docker container seems to require 64bit. At least I don't see any 32bit container available https://hub.docker.com/r/linuxserver/tvheadend/tags

shawly commented 8 months ago

I'm trying to install Docker and TVheadend natively, I'm not trying to use the linuxserver/tvheadend Docker image. Even if I wanted to, docker doesn't work because of the segmentation faults.

tvheadend also provides a Docker image for arm/v6 https://github.com/tvheadend/tvheadend/pkgs/container/tvheadend so that wouldn't be a problem technically.

Joulinar commented 8 months ago

The issue you have on Docker service or the respective container? Is Docker working without the container?

shawly commented 8 months ago

Like I reported, Docker isn't working at all because of segmentation faults

The docker daemon does not start, it does not work

Joulinar commented 8 months ago

Usually this happen if the executable is damaged, corrupted or not able to run on this architecture. But as quite some commands are failing, it looks like a damaged SD card.

shawly commented 8 months ago

Not sure what you mean with quite some commands are failing since it is only a freshly installed docker and tvheadend that cause Illegal instruction errors and segementation faults.

I just tried installing tvheadend=4.3-2155\~gfe4df311d\~bookworm from the http://raspbian.raspberrypi.com/raspbian repository and it does work!

It seems that the latest builds from the tvheadend repos seem to be a problem, as well as the latest docker builds that are installed through dietpi-software. It rather seems that these builds don't work for armv6 anymore?

Joulinar commented 8 months ago

We install Docker from official Docker repository. Not sure if Docker changed something. Might be needed to check with Docker guys if it is able to run on ARMv6 still. Unfortunately more and more apps dropping support for ARMv6.

shawly commented 8 months ago

Yeah it seems like ARMv6 is has been dying for quite some time now, kind of sad for RPi Zero based devices but I guess it is what it is. While tvheadend worked on my RPi 1B+ with the older build, I now switched to usbip and use the Pi simply as host to tunnel my DVB-S2 card to a Docker instance of tvheadend on my home server, which kind of seems to work. So I guess I can live with this setup.

MichaIng commented 8 months ago

I just tested Docker on my RPi Zero: The docker service as well as containerd service start up without errors, and I am able to run the hello-world container:

root@Zero:~# docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
9d2db35bc612: Pull complete
Digest: sha256:d000bc569937abbe195e20322a0bde6b2922d805332fd6d8a68b19f524b7d21d
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (arm32v5)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/get-started/

Hence I am pretty sure that it is one of your containers which is not ARMv6-compatible, not Docker itself, also as it seems to be runc, which fails, which is the final container execution command (not the engine).

Next I tested tvheadend:

root@Zero:~# curl -sSf 'https://dl.cloudsmith.io/public/tvheadend/tvheadend/gpg.C6CC06BD69B430C6.key' | gpg --dearmor -o /etc/apt/trusted.gpg.d/tvheadend.gpg --yes
root@Zero:~# echo 'deb https://dl.cloudsmith.io/public/tvheadend/tvheadend/deb/raspbian bookworm main' > /etc/apt/sources.list.d/tvheadend.list
root@Zero:~# apt update
Hit:1 http://raspbian.raspberrypi.com/raspbian bookworm InRelease
Hit:2 https://download.docker.com/linux/raspbian bookworm InRelease
Hit:3 https://archive.raspberrypi.com/debian bookworm InRelease
Get:4 https://dl.cloudsmith.io/public/tvheadend/tvheadend/deb/raspbian bookworm InRelease [4391 B]
Get:5 https://dl.cloudsmith.io/public/tvheadend/tvheadend/deb/raspbian bookworm/main armhf Packages [13.1 kB]
Fetched 17.5 kB in 6s (2981 B/s)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
All packages are up to date.
root@Zero:~# apt install tvheadend
---
Creating user: hts...
Created symlink /etc/systemd/system/multi-user.target.wants/tvheadend.service → /lib/systemd/system/tvheadend.service.
Could not execute systemctl:  at /usr/bin/deb-systemd-invoke line 145.
Job for tvheadend.service failed because a fatal signal was delivered to the control process.
See "systemctl status tvheadend.service" and "journalctl -xeu tvheadend.service" for details.
invoke-rc.d: initscript tvheadend, action "start" failed.
● tvheadend.service - Tvheadend - a TV streaming server and DVR
     Loaded: loaded (/lib/systemd/system/tvheadend.service; enabled; preset: enabled)
     Active: activating (auto-restart) (Result: signal) since Wed 2024-03-06 16:22:42 CET; 134ms ago
    Process: 3458 ExecStart=/usr/bin/tvheadend -f -p /run/tvheadend.pid $OPTIONS (code=killed, signal=ILL)
        CPU: 75ms
dpkg: error processing package tvheadend (--configure):
 installed tvheadend package post-installation script subprocess returned error exit status 1
Processing triggers for libc-bin (2.36-9+rpt2+deb12u4) ...
Errors were encountered while processing:
 tvheadend
E: Sub-process /usr/bin/dpkg returned an error code (1)
root@Zero:~# journalctl -u tvheadend
Mar 06 16:21:47 Zero systemd[1]: Starting tvheadend.service - Tvheadend - a TV streaming server and DVR...
Mar 06 16:21:47 Zero systemd[1]: tvheadend.service: Control process exited, code=killed, status=4/ILL
Mar 06 16:21:47 Zero systemd[1]: tvheadend.service: Failed with result 'signal'.
Mar 06 16:21:47 Zero systemd[1]: Failed to start tvheadend.service - Tvheadend - a TV streaming server and DVR.
Mar 06 16:22:42 Zero systemd[1]: tvheadend.service: Scheduled restart job, restart counter is at 1.
Mar 06 16:22:42 Zero systemd[1]: Stopped tvheadend.service - Tvheadend - a TV streaming server and DVR.
Mar 06 16:22:42 Zero systemd[1]: Starting tvheadend.service - Tvheadend - a TV streaming server and DVR...
Mar 06 16:22:42 Zero systemd[1]: tvheadend.service: Control process exited, code=killed, status=4/ILL
Mar 06 16:22:42 Zero systemd[1]: tvheadend.service: Failed with result 'signal'.
Mar 06 16:22:42 Zero systemd[1]: Failed to start tvheadend.service - Tvheadend - a TV streaming server and DVR.
root@Zero:~# tvheadend -v
Illegal instruction

So yes, those packages have an issue. I tried some older package versions down to 4.3-2271~g7acca01c4~raspbianbookworm, but all of them fail the same way.

Btw, raspbian means explicitly ARMv6-compatible, so not is not intentional dropped support, but a bug in the build scripts.

I was thinking whether old vs new kernel/firmware might be related, or missing MMAL and other legacy GPU libraries, but OP uses the old (our current stable) packages, while my RPi Zero uses the new one, but both fail the same way.

I'm now testing those on Odroid XU4, just to see whether they really work well on ARMv7. EDIT: Jep, works well on ARMv7:

root@OdroidXU4:~# tvheadend -v
tvheadend: version 4.3-2292~g9ac61d767

EDIT2: Reported here: https://tvheadend.org/d/8554-cloudsmith-raspbian-packages-are-armv6-incompatible

shawly commented 8 months ago

I reinstalled Raspbian Bullseye yesterday and installed Docker from the Documentation and it works flawless now. I think I had previously just followed the Debian bookworm installation guide from Docker until I realized I had to install it for armv6 so I uninstalled it again and used the dietpi-software catalog. I guess there were some leftover packages with the wrong binaries.

I also tried tvheadend again but that indeed has the wrong binaries in the cloudsmith apt repository even for bullseye. Though the package from the raspbian mirrors works fine.

MichaIng commented 8 months ago

Though the package from the raspbian mirrors works fine.

At least there is an alternative then. And the packages from the RPi repo (not Raspbian raspbian.raspberrypi.com, but archive.raspberrypi.com) are not even that old, 4.3-2155~gfe4df311d~bookworm currently.

shawly commented 8 months ago

The packages are also being fixed now https://github.com/tvheadend/tvheadend/issues/1665#issuecomment-1985129929

MichaIng commented 8 months ago

But it is still failing, passing through NEON and ARMv7 flags for some FFmpeg related build steps. This was already the case on the latest available build log from 3 months ago. So I am curious whether this did ever work, or how old the latest ARMv6-compatible tvheadend package from this repository actually is 🤔.

shawly commented 8 months ago

I tried the oldest package from the bookworm and buster tvheadend repo and they both failed with illegal instructions, so they might have never worked at all 🤷.

MichaIng commented 8 months ago

Indeed. It is two FFmpeg/video codec libraries who do not follow parent script build flags but do (too) clever own checks about whether e.g. NEON (an ARMv7-only instruction extension) is supported by the system and compiler or not, which is within the emulated ARMv6 containers used for the builds. I have ideas how to fix it, but which way/method to go with needs to be decided by tvheadend devs.

MichaIng / DietPi