search

easybuilders / easybuild-easyconfigs

A collection of easyconfig files that describe which software to build using which build options with EasyBuild.
https://easybuild.io
GNU General Public License v2.0
380 stars 701 forks source link

GI_TYPELIB_PATH set wrongly #9520

Closed Flamefire closed 4 years ago

Flamefire commented 4 years ago

I tried to use GTK+ with PyGObject and got a failure:

python -c 'import gi; gi.require_version("Gtk", "3.0")' ValueError: Namespace Gtk not available

I executed a find $EBROOTGTKPLUS -name '*.typelib' and it found them under lib/girepository-1.0/Gtk-3.0.typelib. However the ECs set GI_TYPELIB_PATH=share

If I manually change that path in GI_TYPELIB_PATH to $EBROOTGTKPLUS/lib/girepository-1.0 then the above command works.

From a quick grep this seems to affect many ECs.

Flamefire commented 4 years ago

@akesandgren You did that initially in https://github.com/easybuilders/easybuild-easyconfigs/pull/8246. Did you have a testcase? How did you choose share?

akesandgren commented 4 years ago

It fixed a problem with finding Gtk-3.0.gir, which is under share. According to the log of #8246 it had to do with #8241

But judging from the name of the variable I agree that lib/girepos... is where it should be pointing.

But before we change anything we should check what happens with #8241

Flamefire commented 4 years ago

I'm rebuilding #8241 and check that folder

Flamefire commented 4 years ago

It fixed a problem with finding Gtk-3.0.gir, which is under share. According to the log of #8246 it had to do with #8241

Can't find anything in those PRs. Do you know which problem and how to reproduce?

I think the best way to fix this is to set GI_TYPELIB_PATH in EasyBlock. So check for lib/girepo... And add that to module if found. Then remove from ecs and test. Should work.

Btw: can you rebuild a ec starting from a given ec? Basically build Y + dependencies but not X + dependencies

akesandgren commented 4 years ago

After rebuilding WebKitGTK+ with various settings of XDG_DATA_DIRS and GI_TYPELIB_PATH it looks like the real problem in #8241 was not GI_TYPELIB_PATH but for some reason XDG_DATA_DIRS despite it actually being set correctly (unless i had a broken GTK+ build at the time).

So, for any easyconfig using GObject-Introspection as a dependency we should set XDG_DATA_DIRS to 'share' and GI_TYPELIB_PATH to 'lib/girepository-1.0' where 1.0 probably depends on version of GObject-Introspection in some way.

Setting them in each produced module might be possible in the EasyBlock easyblock which most, if not all, easyblocks rely on. However, it won't work if some easyblock skips the super call...

boegel commented 4 years ago

The only common place for all easyblocks is the general EasyBlock class in framework, but I'm not sure that's the right place to define something like GI_TYPELIB_PATH...

What is this for exactly, and for which software packages is this relevant?

Flamefire commented 4 years ago

IMO this is a perfect fit for https://github.com/easybuilders/easybuild-framework/blob/develop/easybuild/framework/easyblock.py#L1316 where similar variables CPATH, PKG_CONFIG_PATH and ACLOCAL_PATH are set. This variable is basically the same but for typelibs.

It is used by GIRepository/GObject and relevant for all packages installing *.typelib files. Most prominent examples are GTK and friends. See https://developer.gnome.org/gi/stable/GIRepository.html

Flamefire commented 4 years ago

After rebuilding WebKitGTK+ with various settings of XDG_DATA_DIRS and GI_TYPELIB_PATH it looks like the real problem in #8241 was not GI_TYPELIB_PATH but for some reason XDG_DATA_DIRS despite it actually being set correctly (unless i had a broken GTK+ build at the time).

Just checked the docs again and they are pretty clear:

https://gi.readthedocs.io/en/latest/tools/g-ir-scanner.html:

The g-ir-scanner uses the XDG_DATA_DIRS variable to check for dirs, the girs are located in XDG_DATA_DIRS/gir-1.0

https://developer.gnome.org/gi/stable/GIRepository.html

GIRepository will typically look for a girepository-1.0 directory under the library directory used when compiling gobject-introspection.

It is possible to control the search paths programmatically, using g_irepository_prepend_search_path(). It is also possible to modify the search paths by using the GI_TYPELIB_PATH environment variable. The environment variable takes precedence over the default search path and the g_irepository_prepend_search_path() calls

So I'll go forward and add a PR and test it on our system.

Flamefire commented 4 years ago

FWIW: I did a grep -rl GI_TYPELIB_PATH easybuild/easyconfigs | xargs grep -L GObject-Introspection and found GST-plugins-base. So the presence of that dependency doesnt seem to the the best indicator. The folder in lib* is better.

I opened https://github.com/easybuilders/easybuild-framework/pull/3133 which does work in my test. Tried: eb --rebuild ATK-2.32.0-GCCcore-8.2.0.eb at-spi2-atk-2.32.0-GCCcore-8.2.0.eb at-spi2-core-2.32.0-GCCcore-8.2.0.eb cairo-1.16.0-GCCcore-8.2.0.eb CMake-3.13.3-GCCcore-8.2.0.eb cURL-7.63.0-GCCcore-8.2.0.eb DBus-1.13.8-GCCcore-8.2.0.eb expat-2.2.6-GCCcore-8.2.0.eb fontconfig-2.13.1-GCCcore-8.2.0.eb freetype-2.9.1-GCCcore-8.2.0.eb FriBidi-1.0.5-GCCcore-8.2.0.eb Gdk-Pixbuf-2.38.1-GCCcore-8.2.0.eb gettext-0.19.8.1-GCCcore-8.2.0.eb GLib-2.60.1-GCCcore-8.2.0.eb GObject-Introspection-1.60.1-GCCcore-8.2.0-Python-3.7.2.eb gperf-3.1-GCCcore-8.2.0.eb GTK+-3.24.8-GCCcore-8.2.0.eb HarfBuzz-2.4.0-GCCcore-8.2.0.eb ICU-64.2-GCCcore-8.2.0.eb intltool-0.51.0-GCCcore-8.2.0.eb libdrm-2.4.97-GCCcore-8.2.0.eb libepoxy-1.5.3-GCCcore-8.2.0.eb libjpeg-turbo-2.0.2-GCCcore-8.2.0.eb libpng-1.6.36-GCCcore-8.2.0.eb LibTIFF-4.0.10-GCCcore-8.2.0.eb libunwind-1.3.1-GCCcore-8.2.0.eb libxml2-2.9.8-GCCcore-8.2.0.eb LLVM-7.0.1-GCCcore-8.2.0.eb Mako-1.0.8-GCCcore-8.2.0.eb Meld-3.20.1-GCCCore-8.2.0-Python-3.7.2.eb Mesa-19.0.1-GCCcore-8.2.0.eb Meson-0.50.0-GCCcore-8.2.0-Python-3.7.2.eb NASM-2.14.02-GCCcore-8.2.0.eb nettle-3.4.1-GCCcore-8.2.0.eb Ninja-1.9.0-GCCcore-8.2.0.eb Pango-1.43.0-GCCcore-8.2.0.eb PCRE-8.43-GCCcore-8.2.0.eb Perl-5.28.1-GCCcore-8.2.0.eb pixman-0.38.0-GCCcore-8.2.0.eb pkg-config-0.29.2-GCCcore-8.2.0.eb PyCairo-1.18.0-GCCcore-8.2.0-Python-3.7.2.eb PyGObject-3.34.0-GCCcore-8.2.0-Python-3.7.2.eb util-linux-2.33-GCCcore-8.2.0.eb X11-20190311-GCCcore-8.2.0.eb and currently running something similar for WebKitGTK+-2.24.1-GCC-8.2.0-2.31.1.eb from #8241

Note: I got those by using the following script which returns the list of ECs that are unique in the dependencies of the given ECs (including those ECs):

function getECs(){
  ECToInstall="$1"
  eb "$ECToInstall" -Dr | grep '^ \*' | cut -c 8- | cut -f 1 -d ' ' | sed 's!.*/!!' | sort -u
}

ECs=""
for EC in "$@"; do
  ECs="$({ echo "$ECs"; getECs "$EC"; })"
done

non_dup_ECs="$(echo "$ECs" | sort | uniq -u)"
echo "$non_dup_ECs"

For the test I removed all XDG_DATA_DIRS and GI_TYPELIB_PATH entries of the modextrapath in all ECs. PR will follow.

Edit: PR for ECs in https://github.com/easybuilders/easybuild-easyconfigs/pull/9528