search

void-linux / void-packages

The Void source packages collection
https://voidlinux.org
Other
2.46k stars 2.09k forks source link

[RFC/tracking issue] deprecating -lib32 multilib system #27337

Open q66 opened 3 years ago

q66 commented 3 years ago

So, right now we have a system in place that works like this:

Proposed new system

Therefore, I am proposing a new system. The goals of this system should be:

There are distributions, mainly Debian, that solve this by having a complete multiarch system in place. That involves having support for handling multiple architectures at once in their package manager, and having everything built in a way that /usr/lib is never used directly, but rather libraries use a special prefix that contains the target triplet, so there are never conflicts. While I considered implementing such system in Void, it quickly turned out to be too large of a can of worms, without that much benefit, which would also require a considerable amount of effort put everywhere. So, instead I am proposing what I call multilib prefixes, which is a simplified, generalized system that only handles the common case of running 32-bit processes on 64-bit systems (or alternatively, 64-bit processes on systems running 64-bit kernel with 32-bit userland).

How do multilib prefixes work?

Before explaining how this is going to work, I'm going to describe the steps needed to get there, it will make it easier. The ones marked so are already done.

Now, to get to how it works, and also why it works.

For one, the /usr/lib path we currently use will be preserved. This is because xbps-src is already updated to intercept anything installing into lib32 or lib64 and properly migrating it to lib using symlinks. This is important.

On 64-bit systems, base-files contains symlinks so that e.g. /usr/lib64 points to /usr/lib. Similar thing happens for lib32 on 32-bit systems. However, since the build systems and so on will automatically default to lib32 or lib64, all library lookups will happen through those symlinks, and not through the actual path. What does this mean?

In order to deal with the case of 32-bit binaries on 64-bit systems, all we have to do is simply create a new Void "root". Let's say this root will be /usr/i686-linux-gnu. This root will contain all the usual files. So your 32-bit libraries will be located in /usr/i686-linux-gnu/usr/lib.

How to install things into this prefix? That is easy; you can just use xbps-install. For example, XBPS_ARCH=i686 xbps-install -r /usr/i686-linux-gnu .... That means you will be able to make use of the entire standard 32-bit repo without any special handling. This is in fact how xbps-src already manages makedepends when cross-compiling.

What is needed now is to hook this prefix into your system. Since 64-bit systems will use lib64 and 32-bit ones will use lib32 for lookups, they are non-conflicting. Therefore, what you can do is make your /usr/lib32 a symlink to /usr/i686-linux-gnu/usr/lib. And that's it; if you run any 32-bit process in your 64-bit system, it will look up its libraries through /usr/lib32, which will point to your custom prefix, containing all the dependencies.

If you think about it, that's not very different from how the current 32bit libs are handled. It's just generalized, and does not use any special tooling.

Of course, this is actually not going to work as is, out of box. There is also the dynamic linker to take care of. This dynamic linker does not always respect lib32 paths or whatever, so it's going to be slightly more involved. Notably on musl, dynamic linker always comes from /lib. E.g. /lib/ld-musl-i686.so.1.

However, the dynamic linkers are the one thing where we are sure there will never be file conflicts. Therefore, you can just symlink the dynamic linker from your multilib prefix to the path where it's expected to be, and everything will just work.

And that is basically the gist of it; there is nothing much more complicated about it conceptually. Of course, there are issues we need to work out, because in practice it's not as smooth.

Problems

There might be more issues to work out, these are just ones I can think of right now. I think all of them are solvable, so we should be able to make a concept like this into reality sometime in near future, and deprecate/remove the current multilib repos.

@void-linux/pkg-committers

ahesford commented 3 years ago

In general this seems like a great idea. Are there any issues with rpaths in this scheme? I believe there are at least a few packages that install executables or shared libs that rely on rpaths to find some dependencies. Of course, if they are hard coded in a way that breaks your proposal, they would probably be broken under the current i686 multilib anyway...

q66 commented 3 years ago

if there are they are already broken in the current scheme, but things with rpaths don't need to concern us much since the whole 32-bit multilib thing is largely for compatibility libraries, it is rare that you'd use it for 32-bit programs (and most of these should still be okay even then)

Skirmisher commented 3 years ago

Seems pretty good all around. Standardizing the existing lib32/64 setup while pointing the multilib component to a separate prefix is a pragmatic approach that still leaves room for flexibility.

I do want to address the prefix location thing: cross toolchain prefixes are weird because they have libraries for the prefix arch in usr/lib, but have host arch binaries in usr/bin, making it inconsistent with a real install (especially since in a multiarch situation, you would be able to run the "native" versions of those tools anyway). I'm not sure if there's a way to work around the host-arch binaries, or if it would be straightforward to move the cross prefixes somewhere else, e.g. /usr/cross/<prefix>... If nothing else, though, I would find it reasonable to put multiarch prefixes in their own subdirectory like /usr/multiarch or /usr/arch or whatever else.

q66 commented 3 years ago

yeah the cross prefix unification thing is just an idea but not something we necessarily have to do

it also has other problems, e.g. gccgo's libgo in crossprefix is built with a dummy static libucontext on musl so it probably does not work, and the libc is built with a partial compiler and sometimes lower optimization level so we likely do not actually want to run anything relying on the crosstoolchain's target libs :P

ericonr commented 3 years ago

We will need tooling for managing these multilib prefixes. Using just xbps as it is is fairly clunky, because by default these prefixes don't have repos set up in them and so on. Especially initial setup is relatively involved.

This initial setup will be rather clunky, since it's not really possible to derive the repo path from whatever is configured in /etc/xbps.d; aarch64 has a completely different path from what arm uses, for example.

Just as a note for something that may not be immediately obvious, the reason we need the /usr/lib32 symlink as well as the dynamic linker configuration, is that the latter affects how programs are loaded (where the linker looks for the libraries your program is linked against), while the former is a fix that will allow libraries like mesa and pulse to simply work, since they will search for things to dlopen in /usr/lib32.

A few concerns that came to mind now were:

There might be packages that require wordsize-specific data files in /usr/share. These need to be found, and such files need to go to /usr/lib(32|64). This is the right thing to do either way, since they're no different from binary files, /usr/share should only contain architecture-independent data.

At least one package that I know of is aspell. EDIT: I was mistaken about aspell, it did put the files in /usr/lib.

ericonr commented 3 years ago

Packages that receive special treatment for i686 (due to multilib) and should be fixed (like https://github.com/void-linux/void-packages/pull/27291) for this project to work (I will be updating this list as I find them; feel free to edit. checks will be added when the package has been fixed and rebuilt):

ericonr commented 3 years ago

Our 32bit nvidia packages are created manually, we'd have to figure out how to make them work.

For this, we could try to build the nvidia stuff inside a i686 masterdir and make packages only with the libraries, without the DKMS module.

ahesford commented 3 years ago

I just did this on an aarch64 installation on a Raspberry Pi, when I realized I couldn't run retroarch because all of the cores precompiled by that project are 32-bit only. Enter multilib forr ARM. Except for the repository path differences (an easy fix), everything worked beautifully.

mkdir -p /usr/armv7l/etc/xbps.d
echo "repository=https://mirrors.servercentral.com/voidlinux/current" > /usr/armv7l/etc/xbps.d/00-repository-main.conf
XBPS_ARCH=armv7l xbps-install -S -C /usr/armv7l/etc/xbps.d retroarch mesa-dri
rm -rf /usr/lib32 && ln -s armv7l/usr/lib /usr/lib32
ln -s /usr/armv7l/usr/lib/ld-2.30.so /usr/lib/ld-linux-armhf.so.3
/usr/armv7l/usr/bin/retroarch

We could wrap this in a simple script that derives XBPS_ARCH from the executable name (e.g., xbps-install-<arch>) which can check for /some/path/to/<arch>, complain if it finds no etc/xbps.d/00-repository-main.conf (or maybe prepopulate the file) and make the necessary links after installation if they are missing.

github-actions[bot] commented 2 years ago

Issues become stale 90 days after last activity and are closed 14 days after that. If this issue is still relevant bump it or assign it.

the-maldridge commented 2 years ago

We'd certainly like to keep this going, I'm going to exempt this from bot actions with the request label, its not quite the right label, but its the quick fix I have right now.