nginx / unit

NGINX Unit - universal web app server - a lightweight and versatile open source server that simplifies the application stack by natively executing application code across eight different programming language runtimes.
https://unit.nginx.org
Apache License 2.0
5.25k stars 320 forks source link

Packaging: fix build-depends on multiarch debian systems #1324

Open thresheek opened 1 week ago

thresheek commented 1 week ago

It's possible to have two versions of the same package installed on debian-based multiarch systems - e.g. i386 alongside amd64. This means that when getting the package status through dpkg-query we'd get a duplicated string:

% dpkg-query -f '$${db:Status-Status}' -W libssl-dev $installed$installed

% dpkg -l | grep libssl-dev ii libssl-dev:amd64 3.0.11-1~deb12u2 amd64 Secure Sockets Layer toolkit - development files ii libssl-dev:i386 3.0.11-1~deb12u2 i386 Secure Sockets Layer toolkit - development files

The fix is to explicitely check for the main architecture and, in case for noarch (or rather all-arch in debian terms) packages, check for special :all architecture as well.

ac000 commented 1 week ago

That's true of Fedora et al as well...

thresheek commented 1 week ago

Indeed, it seems the problem is similar on rpm-based:

% fgrep PRETTY /etc/os-release
PRETTY_NAME="Red Hat Enterprise Linux 9.4 (Plow)"
% rpm -qi --whatprovides openssl-devel | grep -E 'Name|Arch'
Name        : openssl-devel
Architecture: i686
Name        : openssl-devel
Architecture: x86_64

However it's not as easy to fix, since architectures are all over the place:

% linux32 rpm --eval '%_arch'
i386
% rpm -qi --provides openssl-devel.i686 | grep ^openssl-devel\(
openssl-devel(x86-32) = 1:3.0.7-27.el9
% linux32 uname -m
i686

And to make things even more complicated, 700ee28bbfe9b50482e6d34734c856f829bbe23f fixed a real issue on Amazon Linux 2023 with curl-minimal providing curl, php8.2-devel providing php-devel, etc, and they don't really provide a CAPABILITY with the architecture bit there:

% rpm -q --provides curl-minimal
curl = 8.2.1-1.amzn2023.0.3
curl-minimal = 8.2.1-1.amzn2023.0.3
curl-minimal(x86-64) = 8.2.1-1.amzn2023.0.3
ac000 commented 1 week ago

However it's not as easy to fix, since architectures are all over the place:

% linux32 rpm --eval '%_arch'
i386
% rpm -qi --provides openssl-devel.i686 | grep ^openssl-devel\(
openssl-devel(x86-32) = 1:3.0.7-27.el9
% linux32 uname -m
i686

It's been many years since I last needed to run any 32bit stuff (the flash plugin...)

I think part of the issue is that while there is really only x86_64 in terms of AMD/Intel 64bit, there are several subarchs in the 32bit case, (i386, i486, i585 & i686).

From the setarch(8) man-page

The execution domains currently only affects the output of uname -m. For example, on an AMD64 system, running setarch i386 program will cause program to see i686 instead of x86_64 as the machine type.

My guess is that rpm --eval '%_arch' is showing the base architecture, i386 in the case for 32bit Intel.

But and as I say it's been a long time since I needed to use 32bit stuff, I think you'll generally only find i686 rpms these days for 32bit Intel... (in Fedora's case anyway...)

(And of course support for i386 was removed from Linux over a decade ago! ;-))

thresheek commented 1 week ago

The problem is I'm a bit lost on how to detect arch and noarch packages, or rather, Provides which generally are not required to specify the architecture (or something similar to architecture).

E.g. if we use a following patch to query for foo(x86-64) when we have a build-dependency on foo:

diff --git a/pkg/rpm/Makefile b/pkg/rpm/Makefile
index 5d28fb2c..e02796de 100644
--- a/pkg/rpm/Makefile
+++ b/pkg/rpm/Makefile
@@ -176,7 +176,11 @@ check-build-depends-%:
        esac ; \
        not_installed= ; \
        for pkg in $${pkgs}; do \
-               rpm -qi --whatprovides $${pkg} >/dev/null 2>&1 ; \
+               rpmarch="$$(uname -m)" ; \
+               case "$${rpmarch}" in \
+                       x86_64) rpmarch="x86-64";; \
+               esac ; \
+               rpm -qi --whatprovides $${pkg}\($${rpmarch}\) >/dev/null 2>&1 ; \
                if [ $$? -ne 0 ]; then \
                        not_installed="$${not_installed} $${pkg}" ; \
                fi ; \

We will be able to detect e.g. openssl-devel.x86_64 vs openssl-devel.i686, but it will fail on noarch packages, like rpmlint. Also, it will fail on Amazon Linux 2023 with curl-minimal that does provide curl, but does not specify the architecture:

% fgrep -i pretty /etc/os-release
PRETTY_NAME="Amazon Linux 2023"
% rpm -q --provides rpmlint
config(rpmlint) = 1.11-19.amzn2023.0.2
rpmlint = 1.11-19.amzn2023.0.2
% rpm -q --provides curl-minimal
curl = 8.2.1-1.amzn2023.0.3
curl-minimal = 8.2.1-1.amzn2023.0.3
curl-minimal(x86-64) = 8.2.1-1.amzn2023.0.3
ac000 commented 1 week ago

Are you building 32bit packages?

With rpm at least, it defaults to 64bit on x86_64 (as opposed to say defaulting to 32bit on Sparc64).

If you're only building 64bit packages I don't think you really have anything to worry about, I mean whenever I've build rpms in the past I've never needed to worry about such things...

thresheek commented 5 days ago

I'm not building 32 bit packages. The rpm situation described is not something we have on our packages builder machines (contrary to the debian-based ones), but it's still possible for someone who will use our packaging to install both architectures of openssl-devel, which will confuse our check.