gnome: g-ir-scanner scan build error

[tp-m]

I have a feeling this has been discussed somewhere already, but I can't find it right now.

So here's what happened when I did git pull && ninja -C build on a git checkout of gstreamer with a meson build from a few months ago:

[1/346] 'Generating Gst-1.0.gir with a custom command.'
FAILED: gst/Gst-1.0.gir 
'/usr/bin/g-ir-scanner' ../gst/lots-of-c-and-h-files '-pthread' '-I/usr/include/gobject-introspection-1.0' '-I/usr/include/glib-2.0' '-I/usr/lib/x86_64-linux-gnu/glib-2.0/include' '--no-libtool' '--namespace=Gst' '--nsversion=1.0' '--warn-all' '--output' 'gst/Gst-1.0.gir' '--add-init-section=extern void gst_init(gint*,gchar**);g_setenv("GST_REGISTRY_1.0", "/no/way/this/exists.reg", TRUE);g_setenv("GST_PLUGIN_PATH_1_0", "", TRUE);g_setenv("GST_PLUGIN_SYSTEM_PATH_1_0", "", TRUE);gst_init(NULL,NULL);' '--c-include=gst/gst.h' '-I/home/tpm/gst-meson/gstreamer/gst' '-I/home/tpm/gst-meson/gstreamer/build/gst' '-I/home/tpm/gst-meson/gstreamer/build/.' '-I/home/tpm/gst-meson/gstreamer/.' '-I/home/tpm/gst-meson/gstreamer/build/gst/parse' '-I/home/tpm/gst-meson/gstreamer/gst/parse' '--include=GLib-2.0' '--include=GObject-2.0' '--include=GModule-2.0' '--symbol-prefix=gst' '--identifier-prefix=Gst' '--pkg-export=gstreamer-1.0' '.' '-pthread' '-I/usr/include/glib-2.0' '-I/usr/lib/x86_64-linux-gnu/glib-2.0/include' 'parse' '-L/home/tpm/gst-meson/gstreamer/build/gst' '-lunwind' '-lgmodule-2.0' '-pthread' '-lgobject-2.0' '-lm' '-lglib-2.0' '--library' 'gstreamer-1.0'
/home/tpm/gst-meson/gstreamer/build/tmp-introspecttffyaq2m/Gst-1.0: symbol lookup error: /home/tpm/gst-meson/gstreamer/build/tmp-introspecttffyaq2m/Gst-1.0: undefined symbol: gst_dynamic_type_factory_get_type
Command '['/home/tpm/gst-meson/gstreamer/build/tmp-introspecttffyaq2m/Gst-1.0', '--introspect-dump=/home/tpm/gst-meson/gstreamer/build/tmp-introspecttffyaq2m/functions.txt,/home/tpm/gst-meson/gstreamer/build/tmp-introspecttffyaq2m/dump.xml']' returned non-zero exit status 127

In this case there's an older version of the library installed in the install prefix (this particular symbol only appeared in the new version after git pull).

I first thought it's a problem of not using rpath, but when I run ldd on the temp binary it points to the lib in the builddir, and it does have that symbol, so not sure what's going on here. Maybe the runtime path evaluation works differently than ldd.

Here's a way to reproduce it:

#!/bin/bash
set -e

# Make sure the tempdir is not removed on error
export GI_SCANNER_DEBUG="save-temps" 

mkdir -p /tmp/meson-gir-test
cd /tmp/meson-gir-test/

if [ ! -d gstreamer ]; then
  REF="--reference=/home/tpm/gst/master/gstreamer"
  git clone $REF git://anongit.freedesktop.org/gstreamer/gstreamer
fi

cd gstreamer/

# First build and install an older version (latest stable)
git checkout -b local-1.10 origin/1.10 || git checkout local-1.10
if [ -d build ]; then
  rm -rf build;
fi
mkdir build/
cd build/

# FIXME: this might not be the same for everyone!
export LD_LIBRARY_PATH="/tmp/meson-gir-test/prefix/lib/"`uname -m`"-linux-gnu/"

MESON="meson"

$MESON --prefix=/tmp/meson-gir-test/prefix -Ddisable_gtkdoc=true -Ddisable_examples=true
ninja install

# Then update git to a newer version with additional symbols in the library.
# Should not be a problem if the g-ir-scanner uses the locally-built .so from
# the build dir with the new symbols, but bombs out because for some reason
# the old installed library from the prefix is used.
git checkout master
ninja

[tp-m]

Updated script to fix build of old gstreamer version with current meson from master and restore reproducibility. Still fails in the same way.

[thankjura]

meson-0.42.1: can't compile gnome-builder if libgd installed in system:

g-ir-scanner: link: cc -o /var/tmp/portage/gnome-extra/gnome-builder-3.26.0/work/gnome-builder-3.26.0-build/tmp-introspect2nn90ql3/Ide-1.0 -O2 -pipe -march=native -finline-functions -fomit-frame-pointer /var/tmp/portage/gnome-extra/gnome-builder-3.26.0/work/gnome-builder-3.26.0-build/tmp-introspect2nn90ql3/Ide-1.0.o -L. -Wl,-rpath,. -Wl,--no-as-needed -lgd -lide-1.0 -lpangoft2-1.0 -lpango-1.0 -lgobject-2.0 -lglib-2.0 -lfontconfig -lfreetype -lgtksourceview-3.0 -lgtk-3 -lgdk-3 -lpangocairo-1.0 -latk-1.0 -lcairo-gobject -lcairo -lgdk_pixbuf-2.0 -lgio-2.0 -lpeas-1.0 -lgmodule-2.0 -lgirepository-1.0 -ldazzle-1.0 -ltemplate_glib-1.0 -lxml2 -ljson-glib-1.0 -ljsonrpc_glib-1.0 -lwebkit2gtk-4.0 -lsoup-2.4 -ljavascriptcoregtk-4.0 -lm -L/var/tmp/portage/gnome-extra/gnome-builder-3.26.0/work/gnome-builder-3.26.0-build/libide -Wl,-rpath,/var/tmp/portage/gnome-extra/gnome-builder-3.26.0/work/gnome-builder-3.26.0-build/libide -L/usr/lib64/ -Wl,-rpath,/usr/lib64/ -L/var/tmp/portage/gnome-extra/gnome-builder-3.26.0/work/gnome-builder-3.26.0-build/subprojects/libgd/libgd -Wl,-rpath,/var/tmp/portage/gnome-extra/gnome-builder-3.26.0/work/gnome-builder-3.26.0-build/subprojects/libgd/libgd -L/var/tmp/portage/gnome-extra/gnome-builder-3.26.0/work/gnome-builder-3.26.0-build/libide -Wl,-rpath,/var/tmp/portage/gnome-extra/gnome-builder-3.26.0/work/gnome-builder-3.26.0-build/libide -L/var/tmp/portage/gnome-extra/gnome-builder-3.26.0/work/gnome-builder-3.26.0-build/subprojects/libgd/libgd -Wl,-rpath,/var/tmp/portage/gnome-extra/gnome-builder-3.26.0/work/gnome-builder-3.26.0-build/subprojects/libgd/libgd -lgio-2.0 -lgobject-2.0 -Wl,--export-dynamic -lgmodule-2.0 -pthread -lglib-2.0 -Wl,-O1 -Wl,--as-needed

/var/tmp/portage/gnome-extra/gnome-builder-3.26.0/work/gnome-builder-3.26.0-build/libide/libide-1.0.so: undefined reference to `gd_tagged_entry_tag_set_style'

/var/tmp/portage/gnome-extra/gnome-builder-3.26.0/work/gnome-builder-3.26.0-build/libide/libide-1.0.so: undefined reference to `gd_tagged_entry_tag_set_label'

/var/tmp/portage/gnome-extra/gnome-builder-3.26.0/work/gnome-builder-3.26.0-build/libide/libide-1.0.so: undefined reference to `gd_tagged_entry_add_tag'

/var/tmp/portage/gnome-extra/gnome-builder-3.26.0/work/gnome-builder-3.26.0-build/libide/libide-1.0.so: undefined reference to `gd_tagged_entry_tag_new'

/var/tmp/portage/gnome-extra/gnome-builder-3.26.0/work/gnome-builder-3.26.0-build/libide/libide-1.0.so: undefined reference to `gd_tagged_entry_get_type'

/var/tmp/portage/gnome-extra/gnome-builder-3.26.0/work/gnome-builder-3.26.0-build/libide/libide-1.0.so: undefined reference to `gd_tagged_entry_remove_tag'

collect2: error: ld returned 1 exit status

If I remove libgd from system, compile successful

https://bugzilla.gnome.org/show_bug.cgi?id=787578

[nirbheek]

(I updated your comment to be a bit more readable)

This libgd problem is more severe than the 'uses old versions of the same library' problem because it's trying to link to unrelated libraries, so the build will always fail for Gentoo users.

[ebassi]

Still an issue with Meson 0.50.0, though I think it's been fixed in between 2017 and now, and regressed as of a couple of Meson releases.

[andyholmes]

I hit this on meson-0.57.1, not sure about earlier versions though.

[hadess]

Started hitting this with meson-0.62.2-1.fc36.noarch in jhbuild, see https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1382

[ebassi]

The issue is, AFAIR, caused by jhbuild changing the library path and injecting it into the environment in such a way that the introspection scanner ends up getting the system path instead of the jhbuild prefix first.

The only solution I found was to use meson devenv layered on top of jhbuild shell, but these days I stopped using jhbuild as the development environment entirely, and just use a throwaway container; this lets me install stuff into /usr and avoid the issue entirely.

[hadess]

The issue is, AFAIR, caused by jhbuild changing the library path and injecting it into the environment in such a way that the introspection scanner ends up getting the system path instead of the jhbuild prefix first.

By setting LD_LIBRARY_PATH? That hardly seems like something that should cause problems.

[xclaesse]

Ideally g-ir-scanner should not compile/run an executable itself. It should only generate the source file and let Meson/Ninja compile and run it. That would solves tones of issues like this one. I gave a try a while ago, but unfortunately that would require lots of refactoring in gi code.

[xclaesse]

FWIW, I started that work there: https://gitlab.gnome.org/xclaesse/gobject-introspection/-/commit/2bbaedc0e510fba08491b2ff843d13a4bfae7280. IIRC it's mostly blocked by adding support in Meson to preprocess files, which is something we want to add regardless in https://github.com/mesonbuild/meson/pull/10149 and/or https://github.com/mesonbuild/meson/pull/8452.

[ystreet]

The issue is, AFAIR, caused by jhbuild changing the library path and injecting it into the environment in such a way that the introspection scanner ends up getting the system path instead of the jhbuild prefix first.

By setting LD_LIBRARY_PATH? That hardly seems like something that should cause problems.

Yes, gobject-introspection currently needs to run a built binary. If LD_LIBRARY_PATH is not correctly set up, then the system installation of any relevant library may be picked up first before the newly built version.

[hadess]

Yes, gobject-introspection currently needs to run a built binary. If LD_LIBRARY_PATH is not correctly set up, then the system installation of any relevant library may be picked up first before the newly built version.

I would be interested to know what you think an incorrectly setup LD_LIBRARY_PATH would be. I'm guessing that would be an LD_LIBRARY_PATH that isn't manipulated by meson to add the locally built libraries to that LD_LIBRARY_PATH before the paths pointed to by LD_LIBRARY_PATH (and I can understand how that would not be wanted).

I should note that we've talked a lot about how this impacts introspection, but this would also be a problem for any software that relies on running that binary against the newly built libraries, like libfprint generating hwdb rules.

I'm curious to how this could be a g-i or jhbuild specific problem.

[ystreet]

I'm curious to how this could be a g-i or jhbuild specific problem.

This could be g-i specifc because meson doesn't compile the relevant binary, g-i compiles the introspection binary through python's distutils support using arguments passed through to g-ir-scanner: https://github.com/mesonbuild/meson/blob/8059908888fc5c4e9cd9716f180f4a40f3cb6264/mesonbuild/modules/gnome.py#L1155

Then when running the built executable to generate the introspection data, if LD_LIBRARY_PATH points to system libraries before the newly built development libraries, then ld.so will load the system library instead of the built library -> possible undefined symbols error.

Also, this has complications with whether -Wl,rpath,/path sets rpath or runpath in the resulting binary. Quoting from https://amir.rachum.com/blog/2016/09/17/shared-libraries/#rpath-and-runpath

The only difference between rpath and runpath is the order they are searched in. Specifically, their relation to LD_LIBRARY_PATH - rpath is searched in before LD_LIBRARY_PATH while runpath is searched in after. The meaning of this is that rpath cannot be changed dynamically with environment variables while runpath can.

If your compiler/linker is using --enable-new-dtags by default (different distros have different defaults here), then LD_LIBRARY_PATH usage overrides meson's usage of any runpath in the newly built libraries/binaries effectively making ld.so use the older version of said library instead.

[eli-schwartz]

meson devenv happens to set LD_LIBRARY_PATH, though the gnome module doesn't.