search

conda-forge / mesalib-feedstock

A conda-smithy repository for mesalib.
BSD 3-Clause "New" or "Revised" License
2 stars 16 forks source link

`libglapi.so` from mesalib being used in on-screen rendering #59

Open pauldmccarthy opened 9 months ago

pauldmccarthy commented 9 months ago

Solution to issue cannot be found in the documentation.

Issue

Hi @hmaarrfk, I have encountered a prickly issue regarding the libglapi.so library that is bundled with mesalib (I'm currently only considering the Linux build - I suspect that this doesn't affect Windows/macOS).

In some on-screen rendering scenarios, where a program installed into $CONDA_PREFIX is using the system libGL.so library, $CONDA_PREFIX/lib/libglapi.so is being preferentially loaded where the system libglapi.so should be used. Specifically, I am using PyOpenGL - I think that $CONDA_PREFIX/lib/libglapi.so is being used over the system libglapi.so simply because of the RPATH entry baked into $CONDA_PREFIX/bin/python, causing it to prefer libraries in $CONDA_PREFIX/lib over system libraries.

Up until a few days ago, this wasn't a problem (and I had never noticed), as the libglapi.so provided by mesalib was compatible with the version available through most Linux OS package managers.

However, mesa 23.2.1 (published to anaconda.org on 2023-09-29) has had some recent changes, which are beyond my understanding, and which seem to have broken compatibility.

As the mesalib package intends to only provide libOSMesa for off-screen rendering, ideally its libglapi should only be used in off-screen rendering scenarios. So I'm wondering if there would be any problem with statically linking libglapi into libOSMesa.so - in principle, this should possible by setting -Dshared-glapi=disabled in the call to meson setup. Do you have any thoughts on this?

Unfortunately I have been playing with around with this locally, and have discovered that compilation with -Dshared-glapi=disabled is currently not possible - I have opened an issue over at the mesa repository inquiring about this.

Thanks!

As this issue involves OpenGL and non-conda-installed software, I don't have a simple test case. However, the basic issue can be observed by checking which libglapi.so file would be loaded via Python:

$ conda create -c conda-forge -y -p ./test.env python mesalib
$ ./test.env/bin/python -c "import ctypes.util; print(ctypes.util.find_library('glapi'))"
${HOME}/test.env/lib/libglapi.so.0

It is also possible to observe the effect of the recent 23.2.1 mesa release by forcing glxinfo (installed via the system, e.g. mesa-utils on Ubuntu 22.04) to use the mesalib provided libglapi.so. We need to use LD_LIBRARY_PATH or LD_PRELOAD here, as glxinfo is not built via conda-forge, and so will not otherwise preferentially load libraries from $CONDA_PREFIX/lib/:

$ conda create -c conda-forge -y -p ./ml23.1.env python "mesalib=23.1"
$ conda create -c conda-forge -y -p ./ml23.2.env python mesalib
$ LD_LIBRARY_PATH=$(pwd)/ml23.1.env/lib glxinfo
# works fine
$ LD_LIBRARY_PATH=$(pwd)/ml23.2.env/lib glxinfo
# segmentation fault

Installed packages

# packages in environment at ${HOME}/test.env:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.19.1               hd590300_0    conda-forge
ca-certificates           2023.7.22            hbcca054_0    conda-forge
elfutils                  0.189                h6f2b95c_1    conda-forge
gettext                   0.21.1               h27087fc_0    conda-forge
gnutls                    3.7.8                hf3e180e_0    conda-forge
icu                       73.2                 h59595ed_0    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
krb5                      1.21.2               h659d440_0    conda-forge
ld_impl_linux-64          2.40                 h41732ed_0    conda-forge
libarchive                3.7.2                h039dbb9_0    conda-forge
libcurl                   8.3.0                hca28451_0    conda-forge
libdrm                    2.4.114              h166bdaf_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libexpat                  2.5.0                hcb278e6_1    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-ng                 13.2.0               h807b86a_2    conda-forge
libgomp                   13.2.0               h807b86a_2    conda-forge
libiconv                  1.17                 h166bdaf_0    conda-forge
libidn2                   2.3.4                h166bdaf_0    conda-forge
libllvm17                 17.0.1               h5cf9203_0    conda-forge
libmicrohttpd             0.9.77               h97afed2_0    conda-forge
libnghttp2                1.52.0               h61bc06f_0    conda-forge
libnsl                    2.0.0                hd590300_1    conda-forge
libpciaccess              0.17                 h166bdaf_0    conda-forge
libsqlite                 3.43.0               h2797004_0    conda-forge
libssh2                   1.11.0               h0841786_0    conda-forge
libstdcxx-ng              13.2.0               h7e041cc_2    conda-forge
libtasn1                  4.19.0               h166bdaf_0    conda-forge
libunistring              0.9.10               h7f98852_0    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libxcb                    1.15                 h0b41bf4_0    conda-forge
libxml2                   2.11.5               h232c23b_1    conda-forge
libzlib                   1.2.13               hd590300_5    conda-forge
lz4-c                     1.9.4                hcb278e6_0    conda-forge
lzo                       2.10              h516909a_1000    conda-forge
mesalib                   23.2.1               h6b56f8e_0    conda-forge
ncurses                   6.4                  hcb278e6_0    conda-forge
nettle                    3.8.1                hc379101_1    conda-forge
openssl                   3.1.3                hd590300_0    conda-forge
p11-kit                   0.24.1               hc5aa10d_0    conda-forge
pip                       23.2.1             pyhd8ed1ab_0    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
python                    3.12.0          hab00c5b_0_cpython    conda-forge
readline                  8.2                  h8228510_1    conda-forge
setuptools                68.2.2             pyhd8ed1ab_0    conda-forge
tk                        8.6.13               h2797004_0    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
wheel                     0.41.2             pyhd8ed1ab_0    conda-forge
xorg-damageproto          1.2.1             h7f98852_1002    conda-forge
xorg-fixesproto           5.0               h7f98852_1002    conda-forge
xorg-glproto              1.4.17            h7f98852_1002    conda-forge
xorg-kbproto              1.0.7             h7f98852_1002    conda-forge
xorg-libx11               1.8.6                h8ee46fc_0    conda-forge
xorg-libxau               1.0.11               hd590300_0    conda-forge
xorg-libxdamage           1.1.5                h7f98852_1    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xorg-libxext              1.3.4                h0b41bf4_2    conda-forge
xorg-libxfixes            5.0.3             h7f98852_1004    conda-forge
xorg-libxrandr            1.5.2                h7f98852_1    conda-forge
xorg-libxrender           0.9.11               hd590300_0    conda-forge
xorg-randrproto           1.5.0             h7f98852_1001    conda-forge
xorg-renderproto          0.11.1            h7f98852_1002    conda-forge
xorg-util-macros          1.19.3               h7f98852_0    conda-forge
xorg-xextproto            7.3.0             h0b41bf4_1003    conda-forge
xorg-xf86vidmodeproto     2.3.1             h7f98852_1002    conda-forge
xorg-xproto               7.0.31            h7f98852_1007    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
zstd                      1.5.5                hfc55251_0    conda-forge

Environment info

active environment : ${HOME}/test.env
    active env location : ${HOME}/test.env
            shell level : 1
       user config file : ${HOME}/.condarc
 populated config files :
          conda version : 23.1.0
    conda-build version : 3.25.0
         python version : 3.10.12.final.0
       virtual packages : __archspec=1=x86_64
                          __glibc=2.35=0
                          __linux=6.2.0=0
                          __unix=0=0
       base environment : ${HOME}/mamba  (writable)
      conda av data dir : ${HOME}/mamba/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : ${HOME}/mamba/pkgs
                          ${HOME}/.conda/pkgs
       envs directories : ${HOME}/mamba/envs
                          ${HOME}/.conda/envs
               platform : linux-64
             user-agent : conda/23.1.0 requests/2.31.0 CPython/3.10.12 Linux/6.2.0-33-generic ubuntu/22.04.3 glibc/2.35
                UID:GID : 1001:1001
             netrc file : None
           offline mode : False
hmaarrfk commented 9 months ago

Please ping again when you get a resolution upstream.

Can you successfully install an older mesalib on your environent? that might alleviate the issues.

Sounds like you thoroughly checked through things....

pauldmccarthy commented 9 months ago

@hmaarrfk yes, thanks - I'll ping you here if/when I get a response on the mesa issue.

I can work around the problem by pinning mesalib at 23.1, or removing it - it is an optional dependency for my software, so this is by no means a showstopper for me!