glib2: .pc file does not require libpcre2-8

[flyn-org]

Maintainer: @tripolar Environment: x86_64 master

Description:

I have been troubleshooting an issue that prevents building the lang/vala package. The compile fails due to missing libpcre2 symbols. It seems that glib2 is the component that directly depends on libpcre2-8. The problem is that glib2's .pc file does not list libpcre2-8 as a dependency. I edited staging_dir/hostpkg/lib/pkgconfig/glib-2.0.pc, modifying the following line as indicated: Libs: -L${libdir} -lglib-2.0 -pthread -lm -lpcre2-8. This allowed Vala to compile.

I suspect this whole problem arose because OpenWrt statically compiles Vala, which is rare. Static compilation requires careful ordering of -lx arguments, and the Vala build places -lpcre2 too early in the command line (before -lglib2). The .pc change I suggest ensures -lpcre2-8 follows -lglib2.

I am not sure if modifying the .pc file is the right solution. My instinct, though, is to address this in the glib2 package rather than in the vala package, because the pcre2 is a direct dependency of glib2 and only an indirect dependency of vala.

[neheb]

@eli-schwartz know anything about this?

[eli-schwartz]

What are the current contents of the .pc file? It should have a Requires.private: on libpcre2-8 for shared/static builds of glib2, and a Requires for static-only builds of glib2.

[flyn-org]

Here is the .pc file as placed at staging_dir/hostpkg/lib/pkgconfig/glib-2.0.pc by the build process:

prefix=/home/mike/Download/openwrt-sdk-x86-64_gcc-12.3.0_musl.Linux-x86_64/staging_dir/hostpkg
includedir=${prefix}/include
libdir=${prefix}/lib

bindir=${prefix}/bin
glib_genmarshal=${bindir}/glib-genmarshal
gobject_query=${bindir}/gobject-query
glib_mkenums=${bindir}/glib-mkenums

Name: GLib
Description: C Utility Library
Version: 2.74.0
Requires: libpcre2-8 >=  10.32
Libs: -L${libdir} -lglib-2.0 -pthread -lm
Cflags: -I${includedir}/glib-2.0 -pthread -I${libdir}/glib-2.0/include

I modified the Libs line as described above.

[eli-schwartz]

Requires: libpcre2-8 >=  10.32

This should already be enforcing an -lpcre2-8 flag regardless of whether you edit the file or not.

[flyn-org]

Requires: libpcre2-8 >=  10.32

This should already be enforcing an -lpcre2-8 flag regardless of whether you edit the file or not.

I would have thought so, but the Vala build fails without my edit. Something seems wrong.

[VA1DER]

Requires: libpcre2-8 >=  10.32
This should already be enforcing an `-lpcre2-8` flag regardless of whether you edit the file or not.

-lpcre2-8 is on the link line, but it appears too early. It appears before -lglib-2.0 on the command line. Because gcc is anal about library order in some cases, this causes the compilation to fail. Glib's .pc file must itself specify libpcre2-8 as its dependency or else the build system doesn't know to order the command line with -lpcre2-8 AFTER -lglib2.0.

EDIT: This looks, unfortunately, like a bug in glib, as the .pc it is producing is missing the proper dependency.

[eli-schwartz]

Glib's .pc file must itself specify libpcre2-8 as its dependency or else the build system doesn't know to order the command line with -lpcre2-8 AFTER -lglib2.0.

But that pkg-config file demonstrates quite clearly that it is specified as a dependency.

Moreover:

$ pkg-config --libs ./glib-2.0.pc
-L/home/mike/Download/openwrt-sdk-x86-64_gcc-12.3.0_musl.Linux-x86_64/staging_dir/hostpkg/lib -lglib-2.0 -pthread -lm -lpcre2-8

As you can see, -lpcre2-8 appears after -lglib-2.0 on the command line when using your .pc file which you was described above as a paste of the unmodified .pc file.

[VA1DER]

But that pkg-config file demonstrates quite clearly that it is specified as a dependency.

Please reread this post by flyn-org above

The staging_dir/hostpkg/lib/pkgconfig/glib-2.0.pc pkgconfig file that gets installed when it's built does not have -lpcre2-8 as a library dependency: Libs: -L${libdir} -lglib-2.0 -pthread -lm <- This is what it does look like

Which is what causes the problem. Only when you edit staging_dir/hostpkg/lib/pkgconfig/glib-2.0.pc and add -lpcre2-8 as a library dependency of glib will vala compile properly. Like thus: Libs: -L${libdir} -lglib-2.0 -pthread -lm -lpcre2-8 <- This is what it should look like

When this is done, vala knows that -lpcrfe2-8 needs to appear later on its link lines than -lglib-2.0.

[eli-schwartz]

Please reread this post by flyn-org above

Yes, this is exactly what I'm talking about here.

pkgconfig file that gets installed when it's built does not have -lpcre2-8 as a library dependency: Libs: -L${libdir} -lglib-2.0 -pthread -lm <- This is what it does look like

It is not supposed to have a library dependency, a library dependency would be semantically incorrect. It has a recursive pkg-config dependency on libpcre2-8.pc, which file provides the Libs: -lpcre2-8 and is then merged by the pkg-config parser and placed after the direct libraries of glib-2.0.pc in the precise manner which linkers expect.

Which is what causes the problem.

There is no problem with glib's source code or produced artifacts.

[VA1DER]

I'm not sure how I can make this clearer. I am building this as we speak and it's failing. And only when it's changed will it succeed.

Here is the .pc file as placed at staging_dir/hostpkg/lib/pkgconfig/glib-2.0.pc by the build process:

prefix=/home/mike/Download/openwrt-sdk-x86-64_gcc-12.3.0_musl.Linux-x86_64/staging_dir/hostpkg
includedir=${prefix}/include
libdir=${prefix}/lib

bindir=${prefix}/bin
glib_genmarshal=${bindir}/glib-genmarshal
gobject_query=${bindir}/gobject-query
glib_mkenums=${bindir}/glib-mkenums

Name: GLib
Description: C Utility Library
Version: 2.74.0
Requires: libpcre2-8 >=  10.32
Libs: -L${libdir} -lglib-2.0 -pthread -lm
Cflags: -I${includedir}/glib-2.0 -pthread -I${libdir}/glib-2.0/include

THAT .pc, the one quoted above, is the one that OpenWrt's build currently installs into the staging dir, and it doesn't work. It goes on to cause vala's build to fail. It doesn't work the way it is now. Note that -lpcre2-8 is missing from the LIbs line.

It will work if .pc file is changed to be this:

prefix=/home/mike/Download/openwrt-sdk-x86-64_gcc-12.3.0_musl.Linux-x86_64/staging_dir/hostpkg
includedir=${prefix}/include
libdir=${prefix}/lib

bindir=${prefix}/bin
glib_genmarshal=${bindir}/glib-genmarshal
gobject_query=${bindir}/gobject-query
glib_mkenums=${bindir}/glib-mkenums

Name: GLib
Description: C Utility Library
Version: 2.74.0
Requires: libpcre2-8 >=  10.32
Libs: -L${libdir} -lglib-2.0 -pthread -lm -lpcre2-8
Cflags: -I${includedir}/glib-2.0 -pthread -I${libdir}/glib-2.0/include

[eli-schwartz]

I'm not sure how I can make this clearer. I am building this as we speak and it's failing. And only when it's changed will it succeed.

I do not care whether it fails in OpenWrt, I was pinged here for a query about glib2 and I have verified that glib2 is correct.

I in general do not care whether something "fails as we speak", but I do care about the technology behind how something works, and whether it is supposed to fail, and if it does fail when it's not supposed to, the precise manner in which it does so.

Because: "it fails, idk" is not a productive way to debug things, but "it fails in such and such a manner" can often lead you to the root cause of an issue.

I have also verified that pkg-config, when run on the .pc file you quoted, contains the ordering you want, without you touching the Libs line. This is useful and constructive information as far as I'm concerned: it tells me that whatever else the problem may be, one thing that it is not, is a bug in glib, or glib's pkg-config file.

...

I am more than happy to offer advice and suggestions for debugging this and figuring out what the problem is, but we will get nowhere if you continue to erroneously claim that it's a bug in glib2 (and if we cannot move past that erroneous claim, then I'm wasting my time here and might as well unsubscribe).

Some possible things that can help debug the problem:

• You are more than welcome to try the same pkg-config --libs glib-2.0 test I demonstrated and report back what it says.
• It would be interesting to see what GLIB_LIBS is defined to in the vala build.
• It would be interesting to see what exact pkg-config command line is run by the vala configure script.

[VA1DER]

I was pinged here for a query about glib2

Oh, my heavens, I'm so sorry. I thought you were someone else with the OpenWrt team saying there was no issue. Clearly I skimmed way too much.

It would be interesting to see what exact pkg-config command line is run by the vala configure script.

This is a good question - because it's not even an issue of missing --static, since in this .pc libpcre2-8 isn't even a Requires.private, it's straight Requires. The pkg-config line /looks/ good in vala's configure, and it's clearly reading the correct .pc file, or else setting the Libs: line in it would have no effect.

Something hinky is there. I'll dig.

And my apologies again.

[eli-schwartz]

Oh, my heavens, I'm so sorry. I thought you were someone else with the OpenWrt team saying there was no issue. Clearly I skimmed way too much.

Ah, no problem :) it wasn't specifically spelled out anywhere, why I was being pinged.

Something hinky is there. I'll dig.

BTW I cloned vala and poked around in its configure script. It is running:

GLIB_REQUIRED=2.56.0
$PKG_CONFIG --libs "glib-2.0 >= $GLIB_REQUIRED gobject-2.0 >= $GLIB_REQUIRED"

In compiler/Makefile, grep for GLIB_LIBS:

GLIB_LIBS = -pthread -lm -lpcre2-8 -pthread -lgobject-2.0 -lglib-2.0 -pthread -lm -lpcre2-8 -pthread -lffi 

That's on my system -- and also in my libtool linkage for make -C compiler valac.

I haven't thought up a theory for why in your environment there's only a single -lpcre2-8 flag.

[flyn-org]

The discussion between @eli-schwartz and @VA1DER led me to try some more things. Here is the output from pkg-config --libs /home/mike/Download/openwrt-sdk-x86-64_gcc-12.3.0_musl.Linux-x86_64/staging_dir/hostpkg/lib/pkgconfig/glib-2.0.pc on my computer:

-L/home/mike/Download/openwrt-sdk-x86-64_gcc-12.3.0_musl.Linux-x86_64/staging_dir/hostpkg/lib -lglib-2.0 -pthread -lm -lpcre2-8

Note that -lpcre2-8 correctly appears at the end.

Things go wrong somewhere in the OpenWrt build process. Vala's compiler/Makefile in my case contains GLIB_LIBS = -L/home/mike/Download/openwrt-sdk-x86-64_gcc-12.3.0_musl.Linux-x86_64/staging_dir/hostpkg/lib -lgobject-2.0 -lpcre2-8 -lffi -lglib-2.0 -pthread -lm. The -lpcre2-8 is missing.

Here, I try to use OpenWrt's pkg-config rather than my system copy:

$ PKG_CONFIG_PATH=/home/mike/Download/openwrt-sdk-x86-64_gcc-12.3.0_musl.Linux-x86_64/staging_dir/hostpkg/lib/pkgconfig/ ./staging_dir/host/bin/pkg-config.real --libs glib-2.0
-L/home/mike/Download/openwrt-sdk-x86-64_gcc-12.3.0_musl.Linux-x86_64/staging_dir/hostpkg/lib -lglib-2.0 -pthread -lm -lpcre2-8

Things still look good. Now, I try to match the syntax used by the Vala configure script:

$ PKG_CONFIG_PATH=/home/mike/Download/openwrt-sdk-x86-64_gcc-12.3.0_musl.Linux-x86_64/staging_dir/hostpkg/lib/pkgconfig/ ./staging_dir/host/bin/pkg-config.real --libs "glib-2.0 >= 2.48.0 gobject-2.0 >= 2.48.0"
-L/home/mike/Download/openwrt-sdk-x86-64_gcc-12.3.0_musl.Linux-x86_64/staging_dir/hostpkg/lib -lgobject-2.0 -lpcre2-8 -lffi -lglib-2.0 -pthread -lm

The -lpcre2-8 disappeared!

From what I can tell, this is due to the addition of gobject-2.0 >= 2.48.0, as dropping it restores what I need:

PKG_CONFIG_PATH=/home/mike/Download/openwrt-sdk-x86-64_gcc-12.3.0_musl.Linux-x86_64/staging_dir/hostpkg/lib/pkgconfig/ ./staging_dir/host/bin/pkg-config.real --libs "glib-2.0 >= 2.48.0"
-L/home/mike/Download/openwrt-sdk-x86-64_gcc-12.3.0_musl.Linux-x86_64/staging_dir/hostpkg/lib -lglib-2.0 -pthread -lm -lpcre2-8

This is not the case when I use the system pkg-config rather than OpenWrt's pkg-config. The former includes -lpcre2-8 in either case.

Another clue: when using the OpenWrt pkg-config, reordering things as "gobject-2.0 >= 2.48.0 glib-2.0 >= 2.48.0" (i.e., gobject before glib) gives the proper result.

Here are the versions of the two pkg-configs:

$ ./staging_dir/host/bin/pkg-config.real --version
1.9.5
$ pkg-config --version
1.8.0

So, I still have not figured out what is wrong, but I think I am zeroing in on the problem. I am not sure if this is a glib2 packaging problem or vala packaging problem, but I am discussing here rather than at https://github.com/openwrt/packages/issues/21144 because of the context here.

[jefferyto]

This might be related: https://github.com/pkgconf/pkgconf/issues/268

The pkgconf changelog calls 1.9.0 the first testing release of 2.0, and it appears the major distros are sticking to 1.8.

Maybe the main repo shouldn't have updated pkgconf beyond 1.8? Ping @PolynomialDivision

[PolynomialDivision]

Is it fixed when you downgrade again to 1.8?

[flyn-org]

@PolynomialDivision: yes, the build succeeds if I downgrade pkgconf to 1.8.0. Here are the changes I made to do this:

diff --git a/tools/pkgconf/Makefile b/tools/pkgconf/Makefile
index d0a8a271d9..a6659b76c3 100644
--- a/tools/pkgconf/Makefile
+++ b/tools/pkgconf/Makefile
@@ -7,11 +7,11 @@
 include $(TOPDIR)/rules.mk

 PKG_NAME:=pkgconf
-PKG_VERSION:=1.9.5
+PKG_VERSION:=1.8.0

 PKG_SOURCE:=$(PKG_NAME)-$(PKG_VERSION).tar.xz
 PKG_SOURCE_URL:=https://distfiles.dereferenced.org/pkgconf
-PKG_HASH:=1ac1656debb27497563036f7bffc281490f83f9b8457c0d60bcfb638fb6b6171
+PKG_HASH:=ef9c7e61822b7cb8356e6e9e1dca58d9556f3200d78acab35e4347e9d4c2bbaf

 PKG_CPE_ID:=cpe:/a:pkgconf:pkgconf

@@ -24,7 +24,7 @@ HOSTCC := $(HOSTCC_NOCACHE)

 MESON_HOST_ARGS += \
        -Ddefault_library=static \
-       -Dtests=disabled
+       -Dtests=false

 define Host/Install
        $(call Host/Install/Meson)

Should we downgrade pkgconf, or should I patch around this in the vala package?

[flyn-org]

I submitted a pull request that applies a circumvention patch for the vala package. See https://github.com/openwrt/packages/pull/21434.

[PolynomialDivision]

Sorry, I am currently traveling without my laptop.

Is vala the only package failing?