raspberrypi / userland

Source code for ARM side libraries for interfacing to Raspberry Pi GPU.
BSD 3-Clause "New" or "Revised" License
2.05k stars 1.09k forks source link

[Bullseye] Add libraspberrypi0 symlinks to package #715

Closed MichaIng closed 2 years ago

MichaIng commented 2 years ago

Describe the bug The recent Bullseye version of the libraspberrypi0 package ships with library file names with .0 appended, compared to earlier versions of the package on Bullseye and all of them on Buster. This change broke all binaries which were linked against the old names. In the meantime, not all but most packages shipped by the RPi repo have been updated to be linked against the new names, but for 3rd party builds this is by far not the case yet, so we and 3rd party software developers still face a high number of reports about failing binary executions due to revised shared object links.

The problem is now that the official RPi OS Bullseye images ships with symlink from old to new names, obviously created manually on those images, which restores backwards compatibility to old binaries. The problem is that now when developers are compiling software on these images, it will be again linked to the old names, hence won't solve the issue for users. And by far not all systems are guaranteed to have the symlinks:

I think a choice need to me made whether the symlink/old names are generally expected to be present or not. If so, then adding them to the libraspberrypi0 package is the only right solution.

XECDesign commented 2 years ago

You can install libraspberrypi-dev (https://www.debian.org/doc/debian-policy/ch-sharedlibs.html). Otherwise, if people need old binaries to work and don't want to install the dev package, buster remains an option.

The problem is now that the official RPi OS Bullseye images ships with symlink from old to new names, obviously created manually on those images

They're not created manually. They come from the -dev package, added there automatically by dh.

The problem is that now when developers are compiling software on these images, it will be again linked to the old names, hence won't solve the issue for users. And by far not all systems are guaranteed to have the symlinks

When you link a program against a symlink, the actual dependency you should see when you run ldd on the binary would be the .so.SOVERSION file.

MichaIng commented 2 years ago

You can install libraspberrypi-dev

Ah I see. However, it shouldn't be necessary to have development headers installed to execute a binary. Reading the Debian policy that you reasonably follow, it's not clear to me where the issue then is:

Otherwise, if people need old binaries to work and don't want to install the dev package, buster remains an option.

Staying on Buster only for running old binaries when all you need is some symlinks? Not an option 😄, here is what one should do when facing the issue:

cd /usr/lib/arm-linux-gnueabihf
while read -r line
do
[[ ! -f $line || -f ${line%.0} ]] && continue
line=${line#/usr/lib/arm-linux-gnueabihf/}
ln -sf "$line" "${line%.0}"
done < <(dpkg -L 'libraspberrypi0' | grep '^/usr/lib/arm-linux-gnueabihf/.*\.so.0$')

When you link a program against a symlink, the actual dependency you should see when you run ldd on the binary would be the .so.SOVERSION file.

Hmm, this contradicts what we see with RPi Cam Control Interface binaries: The symlinks were there and the binaries were linked against paths without the .0 ending. I'll verify.

XECDesign commented 2 years ago

What software are people normally reporting issues with?

However, it shouldn't be necessary to have development headers installed to execute a binary.

Sure, but I'd say that software should be rebuilt to link against the new libraries. patchelf can be used where the source is not available for whatever reason.

  • First of all most library packages shipped by the official Debian repository ship .so only files or symlinks, sometimes with .so linking to .so.X, sometimes the other way round, so the rule to have .so in development packages only isn't a strict one. glibc itself is shipped as .so.

On my pi there are 27 runtime packages containing .so files in the standard LD library path compared to 331 development packages. So yes, there are a few exceptions, but they generally have their version already somewhere else in the filename or aren't things you'd link against directly.

  • More importantly, ALL Debian binary packages ship executables which can run without any development package installed. While for the library names there is no consistency, a strict consistency is the clear separation between library packages for executing binaries and development packages only for compiling binaries.

And that's where we need to get to as well. I'm aware that we ship chromium built with the old filenames, but that will be fixed.

MichaIng commented 2 years ago

What software are people normally reporting issues with?

Basically with everything that uses this library. Due to intense reports upstream whenever faced, many cases are solved, but e.g. even the RealVNC package shipped by the RPi repo is still pending: https://github.com/RPi-Distro/repo/issues/267 Chromium has been solved/rebuilt, if I'm not mistaken.

Another remaining prominent case is Jellyfin, respectively its dedicated FFmpeg package: https://github.com/jellyfin/jellyfin-ffmpeg/issues/75

And as linked RPi Cam Control, where the binary seems to have been linked against .so even that those were symlink to .so.0: https://github.com/silvanmelchior/RPi_Cam_Web_Interface/issues/640 Not sure whether all toolchains/linkers are supposed to resolve/follow symlink. If it turns out to be a toolchain issue, then I'd say it's fine to leave things as they are as earlier or later the remaining cases will be solved with rebuilds.

XECDesign commented 2 years ago

There could be something strange with the build environment for RPi Cam Control. Perhaps it's built on buster or with a run of rpi-update. But in general, all I needed to do for software we have control over was just rebuild it (and everything would've had the -dev package present).

Chromium has been solved/rebuilt, if I'm not mistaken.

Ah, I'm probably thinking of the gtk dependency change then.

MichaIng commented 2 years ago

Ah, I'm probably thinking of the gtk dependency change then.

Ah right, that is still missing, and the lost ARMv6 (RPi 1/Zero (1)) compatibility 😢.

There could be something strange with the build environment for RPi Cam Control.

He compiled new binaries on a fresh Raspberry Pi OS Bullseye image, I'm waiting for reply whether ldd now shows the correct paths.

XECDesign commented 2 years ago

the lost ARMv6 (RPi 1/Zero (1)) compatibility cry.

I don't think that's coming back, unfortunately.

MichaIng commented 2 years ago

Probably not such a big issue, considering the expected bad browser experience on those models. Even on RPi 2 it is not awesome 😄.

XECDesign commented 2 years ago

I think this one can be closed, since there doesn't seem to be a problem with the packaging. Other software behaves as intended.

MichaIng commented 2 years ago

Last reply on RPi Cam Web Interface (raspimjpeg actually) was still resulting in linking to the symlinks, however, it is then probably indeed an issue with the used toolchain. As the libraspberrypi-dev package with symlinks doesn't have the same effect on other builds, obviously, I'll close.