conda-forge / llvmlite-feedstock

A conda-smithy repository for llvmlite.
BSD 3-Clause "New" or "Revised" License
0 stars 20 forks source link

"LLVM ERROR: inconsistency in registered CommandLine options" on Ubuntu when importing numba within QGIS #54

Closed danschef closed 3 years ago

danschef commented 3 years ago

I initially filed this issue in the numba-feedstock (see here). However, @henryiii pointed me to LLVM, so I re-open the issue here:

Issue:

I am currently involved in the development of a QGIS plugin that uses numba under the hood (requires llvmlite) and there seems to be a compatibility issue between numba and QGIS on Linux (at least on my Ubuntu 20.04 machine).

When installing qgis and numba from conda-forge into the same environment, qgis crashes when trying to import numba via the QGIS Python console. Here is a minimal environment that reproduces the issue:

$ conda create -n qgisnumba -c conda-forge qgis numba
This installs the following environment (conda list):

``` $ conda list # packages in environment at /home/danschef/miniconda3/envs/qgisnumba: # # Name Version Build Channel _libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 1_gnu conda-forge boost-cpp 1.72.0 h9d3c048_4 conda-forge brotlipy 0.7.0 py39h3811e60_1001 conda-forge bzip2 1.0.8 h7f98852_4 conda-forge c-ares 1.17.1 h7f98852_1 conda-forge ca-certificates 2021.1.19 h06a4308_1 cairo 1.16.0 h7979940_1007 conda-forge certifi 2020.12.5 py39hf3d152e_1 conda-forge cffi 1.14.5 py39he32792d_0 conda-forge cfitsio 3.470 hb418390_7 conda-forge chardet 4.0.0 py39hf3d152e_1 conda-forge coverage 5.5 py39h3811e60_0 conda-forge cryptography 3.4.6 py39hbca0aa6_0 conda-forge curl 7.75.0 h979ede3_0 conda-forge dbus 1.13.18 hb2f20db_0 exiv2 0.27.1 had08079_0 conda-forge expat 2.2.10 h9c3ff4c_0 conda-forge fontconfig 2.13.1 hba837de_1004 conda-forge freetype 2.10.4 h0708190_1 conda-forge freexl 1.0.6 h7f98852_0 conda-forge future 0.18.2 py39hf3d152e_3 conda-forge gdal 3.2.1 py39h409cc32_7 conda-forge geos 3.9.1 h9c3ff4c_2 conda-forge geotiff 1.6.0 h11d48b3_4 conda-forge gettext 0.19.8.1 h0b5b191_1005 conda-forge giflib 5.2.1 h516909a_2 conda-forge glib 2.66.7 h9c3ff4c_1 conda-forge glib-tools 2.66.7 h9c3ff4c_1 conda-forge gsl 2.6 he838d99_2 conda-forge gst-plugins-base 1.18.4 h29181c9_0 conda-forge gstreamer 1.18.4 h76c114f_0 conda-forge hdf4 4.2.13 h10796ff_1004 conda-forge hdf5 1.10.6 nompi_h6a2412b_1114 conda-forge httplib2 0.19.0 pyhd8ed1ab_0 conda-forge icu 68.1 h58526e2_0 conda-forge idna 2.10 pyh9f0ad1d_0 conda-forge jinja2 2.11.3 pyh44b312d_0 conda-forge jpeg 9d h516909a_0 conda-forge json-c 0.15 h98cffda_0 conda-forge jsoncpp 1.9.4 h4bd325d_1 conda-forge kealib 1.4.14 hcc255d8_2 conda-forge krb5 1.17.2 h926e7f8_0 conda-forge laszip 3.4.3 he1b5a44_1 conda-forge laz-perf 1.5.0 he1b5a44_0 conda-forge ld_impl_linux-64 2.35.1 hea4e1c9_2 conda-forge libblas 3.9.0 8_openblas conda-forge libcblas 3.9.0 8_openblas conda-forge libclang 11.1.0 default_ha53f305_0 conda-forge libcurl 7.75.0 hc4aaa36_0 conda-forge libdap4 3.20.6 hd7c4107_1 conda-forge libedit 3.1.20210216 h27cfd23_1 libev 4.33 h516909a_1 conda-forge libevent 2.1.10 hcdb4288_3 conda-forge libffi 3.3 h58526e2_2 conda-forge libgcc-ng 9.3.0 h2828fa1_18 conda-forge libgdal 3.2.1 h38ff51b_7 conda-forge libgfortran-ng 9.3.0 hff62375_18 conda-forge libgfortran5 9.3.0 hff62375_18 conda-forge libglib 2.66.7 h3e27bee_1 conda-forge libgomp 9.3.0 h2828fa1_18 conda-forge libiconv 1.16 h516909a_0 conda-forge libkml 1.3.0 hd79254b_1012 conda-forge liblapack 3.9.0 8_openblas conda-forge libllvm10 10.0.1 he513fc3_3 conda-forge libllvm11 11.1.0 hf817b99_0 conda-forge libnetcdf 4.7.4 nompi_h56d31a8_107 conda-forge libnghttp2 1.43.0 h812cca2_0 conda-forge libopenblas 0.3.12 pthreads_h4812303_1 conda-forge libpng 1.6.37 hed695b0_2 conda-forge libpq 13.1 hfd2b0eb_2 conda-forge libprotobuf 3.15.6 h780b84a_0 conda-forge librttopo 1.1.0 h1185371_6 conda-forge libspatialindex 1.9.3 he1b5a44_3 conda-forge libspatialite 5.0.1 he52d314_3 conda-forge libssh2 1.9.0 ha56f1ee_6 conda-forge libstdcxx-ng 9.3.0 h6de172a_18 conda-forge libtiff 4.2.0 hdc55705_0 conda-forge libuuid 2.32.1 h14c3975_1000 conda-forge libwebp 1.2.0 h3452ae3_0 conda-forge libwebp-base 1.2.0 h7f98852_1 conda-forge libxcb 1.14 h7b6447c_0 libxkbcommon 1.0.3 he3ba5ed_0 conda-forge libxml2 2.9.10 h72842e0_3 conda-forge libxslt 1.1.33 h15afd5d_2 conda-forge libzip 1.7.3 he9f05b3_0 conda-forge llvmlite 0.36.0 py39h1bbdace_0 conda-forge lz4-c 1.9.3 h9c3ff4c_0 conda-forge markupsafe 1.1.1 py39h3811e60_3 conda-forge mock 4.0.3 py39hf3d152e_1 conda-forge mysql-common 8.0.23 ha770c72_1 conda-forge mysql-libs 8.0.23 h935591d_1 conda-forge ncurses 6.2 h58526e2_4 conda-forge nitro 2.7.dev6 he1b5a44_3 conda-forge nose2 0.9.2 py_0 conda-forge nspr 4.30 h9c3ff4c_0 conda-forge nss 3.62 hb5efdd6_0 conda-forge numba 0.53.0 py39h4292f66_0 conda-forge numpy 1.20.1 py39hdbf815f_0 conda-forge openjpeg 2.4.0 hf7af979_0 conda-forge openssl 1.1.1j h7f98852_0 conda-forge owslib 0.23.0 pyhd8ed1ab_0 conda-forge pcre 8.44 he1b5a44_0 conda-forge pdal 2.2.0 h638e970_7 conda-forge pip 21.0.1 pyhd8ed1ab_0 conda-forge pixman 0.40.0 h36c2ea0_0 conda-forge plotly 4.14.3 pyh44b312d_0 conda-forge poppler 0.89.0 h2de54a5_5 conda-forge poppler-data 0.4.10 0 conda-forge postgresql 13.1 h6303168_2 conda-forge proj 7.2.0 h277dcde_2 conda-forge psycopg2 2.8.6 py39h3811e60_2 conda-forge pycparser 2.20 pyh9f0ad1d_2 conda-forge pygments 2.8.1 pyhd8ed1ab_0 conda-forge pyopenssl 20.0.1 pyhd8ed1ab_0 conda-forge pyparsing 2.4.7 pyh9f0ad1d_0 conda-forge pyproj 3.0.1 py39h50a64a9_0 conda-forge pyqt 5.12.3 py39hf3d152e_7 conda-forge pyqt-impl 5.12.3 py39h0fcd23e_7 conda-forge pyqt5-sip 4.19.18 py39he80948d_7 conda-forge pyqtchart 5.12 py39h0fcd23e_7 conda-forge pyqtwebengine 5.12.1 py39h0fcd23e_7 conda-forge pyqtwebkit 5.212 py39h20c28b1_1 conda-forge pysocks 1.7.1 py39hf3d152e_3 conda-forge python 3.9.2 hffdb5ce_0_cpython conda-forge python-dateutil 2.8.1 py_0 conda-forge python_abi 3.9 1_cp39 conda-forge pytz 2021.1 pyhd8ed1ab_0 conda-forge pyyaml 5.4.1 py39h3811e60_0 conda-forge qca 2.2.1 h73816c6_3 conda-forge qgis 3.18.0 py39hce95e4c_3 conda-forge qjson 0.9.0 h73816c6_1006 conda-forge qscintilla2 2.11.2 py39hf01f197_4 conda-forge qt 5.12.9 hda022c4_4 conda-forge qtkeychain 0.12.0 h2264404_0 conda-forge qtlocation 5.12.9 he1b5a44_0 conda-forge qtserialport 5.9.8 h73816c6_1 conda-forge qtwebkit 5.212 h8f65c2e_1 conda-forge qwt 6.1.6 h7ec6b3e_0 conda-forge qwtpolar 1.1.1 h73816c6_7 conda-forge readline 8.1 h27cfd23_0 requests 2.25.1 pyhd3deb0d_0 conda-forge retrying 1.3.3 py_2 conda-forge setuptools 52.0.0 py39h06a4308_0 six 1.15.0 pyh9f0ad1d_0 conda-forge sqlite 3.35.1 hdfb4753_0 tiledb 2.2.4 h91fcb0e_3 conda-forge tk 8.6.10 hed695b0_1 conda-forge tzcode 2021a h7f98852_1 conda-forge tzdata 2021a he74cb21_0 conda-forge urllib3 1.26.4 pyhd8ed1ab_0 conda-forge wheel 0.36.2 pyhd3deb0d_0 conda-forge xerces-c 3.2.3 h9d8b166_2 conda-forge xorg-kbproto 1.0.7 h14c3975_1002 conda-forge xorg-libice 1.0.10 h516909a_0 conda-forge xorg-libsm 1.2.3 hd9c2040_1000 conda-forge xorg-libx11 1.7.0 h36c2ea0_0 conda-forge xorg-libxext 1.3.4 h7f98852_1 conda-forge xorg-libxrender 0.9.10 h7f98852_1003 conda-forge xorg-renderproto 0.11.1 h14c3975_1002 conda-forge xorg-xextproto 7.3.0 h14c3975_1002 conda-forge xorg-xproto 7.0.31 h14c3975_1007 conda-forge xz 5.2.5 h516909a_1 conda-forge yaml 0.2.5 h516909a_0 conda-forge zlib 1.2.11 h516909a_1010 conda-forge zstd 1.4.9 ha95c52a_0 conda-forge ```

QGIS can than be started by running qgis on the command line and everything works fine. However, as soon as I open a Python console within QGIS and import numba, QGIS crashes and raises the following error at the command line:

(qgisnumba) gfz-fe@geoms:~$ qgis
Warning: QXcbConnection: XCB error: 145 (Unknown), sequence: 179, resource id: 0, major code: 139 (Unknown), minor code: 20
: CommandLine Error: Option 'help-list' registered more than once!
LLVM ERROR: inconsistency in registered CommandLine options
QGIS died on signal 11Aborted

Note that when I start a Python console directly on the command line (without QGIS), I can import qgis and numba without any issues. So the error only occurs when numba is imported from the QGIS command line (or by a QGIS plugin which uses numba). We also tested this with the same environment on Windows, without any issues.


Details about conda and system ( conda info ):

``` $ conda info (qgisnumba) gfz-fe@geoms:~$ conda info active environment : qgisnumba active env location : /home/danschef/miniconda3/envs/qgisnumba shell level : 4 user config file : /home/danschef/.condarc populated config files : /home/danschef/.condarc conda version : 4.9.2 conda-build version : 3.21.4 python version : 3.8.8.final.0 virtual packages : __glibc=2.31=0 __unix=0=0 __archspec=1=x86_64 base environment : /home/danschef/miniconda3 (writable) channel URLs : file:///home/danschef/.conda_channels_local/pygfz/linux-64 file:///home/danschef/.conda_channels_local/pygfz/noarch https://conda.anaconda.org/conda-forge/linux-64 https://conda.anaconda.org/conda-forge/noarch https://repo.anaconda.com/pkgs/main/linux-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/r/linux-64 https://repo.anaconda.com/pkgs/r/noarch package cache : /home/danschef/miniconda3/pkgs /home/danschef/.conda/pkgs envs directories : /home/danschef/miniconda3/envs /home/danschef/.conda/envs platform : linux-64 user-agent : conda/4.9.2 requests/2.25.1 CPython/3.8.8 Linux/5.4.0-52-generic ubuntu/20.04.1 glibc/2.31 UID:GID : 1009:38755 netrc file : None offline mode : False ```
chrisburr commented 3 years ago

This is an issue for root as well. Getting llvmlite from pip avoids the issue so I suspect it's an issue with how llvmlite is dynamically linking to LLVM.

chrisburr commented 3 years ago

Actually it's not that simple, it's still a problem if I switch to static linking by removing this line:

https://github.com/conda-forge/llvmlite-feedstock/blob/4b12b7e3eb516f7ddb86b7d11a8b256e724c1807/recipe/build.sh#L8

chrisburr commented 3 years ago

I can get a working binary if I:

The CMake build can be fixed by applying:

diff --git a/ffi/CMakeLists.txt.old b/ffi/CMakeLists.txt
old mode 100644
new mode 100755
index d87735c..59f1b4b
--- a/ffi/CMakeLists.txt.old
+++ b/ffi/CMakeLists.txt
@@ -61,3 +61,4 @@ list(REMOVE_ITEM llvm_libs "OptRemarks")

 # Link against LLVM libraries
 target_link_libraries(llvmlite ${llvm_libs})
+set_property(TARGET llvmlite APPEND_STRING PROPERTY LINK_FLAGS "-Wl,--exclude-libs,ALL")
xhochy commented 3 years ago

Probably something in the qgis stack is statically linking LLVM and doesn't use a (correct) version script. Probably switching that component to dynamic LLVM linkage should help here.

chrisburr commented 3 years ago

Yup, that was the problem for root (static linking is essential in this case but the symbols weren't being hidden correctly).

Creating the environment and greping for symbols I don't notice anything suspicious. My best guess would be that it's an OpenGL issue on @danschef's machine with the graphics card driver linking to LLVM without hiding the symbols. Similar to the bug being reported here:

IIRC export LIBGL_ALWAYS_INDIRECT=1 can help in some cases as a workaround for the buggy drivers.

danschef commented 3 years ago

@chrisburr Thanks a lot for looking into this! I just installed qgis + numba again on another machine (Kubuntu 17.04) and there is no error when importing numba within the QGIS Python console. So, at least, this does not seem to be a Linux issue in general. Is there anything I can do to make sure it is not an OpenGL issue only on my machine as you guessed?

Here is the output of conda list on that second machine:

conda list ``` # packages in environment at /home/danschef/mambaforge/envs/qgisnumba: # # Name Version Build Channel _libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 1_gnu conda-forge boost-cpp 1.74.0 hc6e9bd1_2 conda-forge brotlipy 0.7.0 py39h3811e60_1001 conda-forge bzip2 1.0.8 h7f98852_4 conda-forge c-ares 1.17.1 h7f98852_1 conda-forge ca-certificates 2020.12.5 ha878542_0 conda-forge cairo 1.16.0 h6cf1ce9_1008 conda-forge certifi 2020.12.5 py39hf3d152e_1 conda-forge cffi 1.14.5 py39he32792d_0 conda-forge cfitsio 3.470 hb418390_7 conda-forge chardet 4.0.0 py39hf3d152e_1 conda-forge coverage 5.5 py39h3811e60_0 conda-forge cryptography 3.4.7 py39hbca0aa6_0 conda-forge curl 7.76.1 h979ede3_1 conda-forge dbus 1.13.6 h48d8840_2 conda-forge exiv2 0.27.1 had08079_0 conda-forge expat 2.3.0 h9c3ff4c_0 conda-forge fontconfig 2.13.1 hba837de_1005 conda-forge freetype 2.10.4 h0708190_1 conda-forge freexl 1.0.6 h7f98852_0 conda-forge future 0.18.2 py39hf3d152e_3 conda-forge gdal 3.2.2 py39h409cc32_3 conda-forge geos 3.9.1 h9c3ff4c_2 conda-forge geotiff 1.6.0 hcf90da6_5 conda-forge gettext 0.19.8.1 h0b5b191_1005 conda-forge giflib 5.2.1 h516909a_2 conda-forge glib 2.68.1 h9c3ff4c_0 conda-forge glib-tools 2.68.1 h9c3ff4c_0 conda-forge gsl 2.6 he838d99_2 conda-forge gst-plugins-base 1.18.4 h29181c9_0 conda-forge gstreamer 1.18.4 h76c114f_0 conda-forge hdf4 4.2.13 h10796ff_1005 conda-forge hdf5 1.10.6 nompi_h6a2412b_1114 conda-forge httplib2 0.19.1 pyhd8ed1ab_0 conda-forge icu 68.1 h58526e2_0 conda-forge idna 2.10 pyh9f0ad1d_0 conda-forge jinja2 2.11.3 pyh44b312d_0 conda-forge jpeg 9d h516909a_0 conda-forge json-c 0.15 h98cffda_0 conda-forge jsoncpp 1.9.4 h4bd325d_2 conda-forge kealib 1.4.14 hcc255d8_2 conda-forge krb5 1.17.2 h926e7f8_0 conda-forge laszip 3.4.3 he1b5a44_1 conda-forge laz-perf 1.5.0 he1b5a44_0 conda-forge ld_impl_linux-64 2.35.1 hea4e1c9_2 conda-forge libblas 3.9.0 8_openblas conda-forge libcblas 3.9.0 8_openblas conda-forge libclang 11.1.0 default_ha53f305_0 conda-forge libcurl 7.76.1 hc4aaa36_1 conda-forge libdap4 3.20.6 hd7c4107_2 conda-forge libedit 3.1.20191231 he28a2e2_2 conda-forge libev 4.33 h516909a_1 conda-forge libevent 2.1.10 hcdb4288_3 conda-forge libffi 3.3 h58526e2_2 conda-forge libgcc-ng 9.3.0 h2828fa1_19 conda-forge libgdal 3.2.2 hbf32332_3 conda-forge libgfortran-ng 9.3.0 hff62375_19 conda-forge libgfortran5 9.3.0 hff62375_19 conda-forge libglib 2.68.1 h3e27bee_0 conda-forge libgomp 9.3.0 h2828fa1_19 conda-forge libiconv 1.16 h516909a_0 conda-forge libkml 1.3.0 h238a007_1013 conda-forge liblapack 3.9.0 8_openblas conda-forge libllvm10 10.0.1 he513fc3_3 conda-forge libllvm11 11.1.0 hf817b99_2 conda-forge libnetcdf 4.8.0 nompi_hfa85936_101 conda-forge libnghttp2 1.43.0 h812cca2_0 conda-forge libopenblas 0.3.12 pthreads_h4812303_1 conda-forge libpng 1.6.37 hed695b0_2 conda-forge libpq 13.1 hfd2b0eb_2 conda-forge libprotobuf 3.15.8 h780b84a_0 conda-forge librttopo 1.1.0 h1185371_6 conda-forge libspatialindex 1.9.3 he1b5a44_3 conda-forge libspatialite 5.0.1 h20cb978_4 conda-forge libssh2 1.9.0 ha56f1ee_6 conda-forge libstdcxx-ng 9.3.0 h6de172a_19 conda-forge libtiff 4.2.0 hdc55705_0 conda-forge libuuid 2.32.1 h14c3975_1000 conda-forge libwebp 1.2.0 h3452ae3_0 conda-forge libwebp-base 1.2.0 h7f98852_2 conda-forge libxcb 1.13 h7f98852_1003 conda-forge libxkbcommon 1.0.3 he3ba5ed_0 conda-forge libxml2 2.9.10 h72842e0_4 conda-forge libxslt 1.1.33 h15afd5d_2 conda-forge libzip 1.7.3 he9f05b3_0 conda-forge llvmlite 0.36.0 py39h1bbdace_0 conda-forge lz4-c 1.9.3 h9c3ff4c_0 conda-forge markupsafe 1.1.1 py39h3811e60_3 conda-forge mock 4.0.3 py39hf3d152e_1 conda-forge mysql-common 8.0.23 ha770c72_1 conda-forge mysql-libs 8.0.23 h935591d_1 conda-forge ncurses 6.2 h58526e2_4 conda-forge nitro 2.7.dev6 he1b5a44_3 conda-forge nose2 0.9.2 py_0 conda-forge nspr 4.30 h9c3ff4c_0 conda-forge nss 3.64 hb5efdd6_0 conda-forge numba 0.53.1 py39h4292f66_0 conda-forge numpy 1.20.2 py39hdbf815f_0 conda-forge openjpeg 2.4.0 hf7af979_0 conda-forge openssl 1.1.1k h7f98852_0 conda-forge owslib 0.23.0 pyhd8ed1ab_0 conda-forge pcre 8.44 he1b5a44_0 conda-forge pdal 2.2.0 h638e970_7 conda-forge pip 21.0.1 pyhd8ed1ab_0 conda-forge pixman 0.40.0 h36c2ea0_0 conda-forge plotly 4.14.3 pyh44b312d_0 conda-forge poppler 21.03.0 h93df280_0 conda-forge poppler-data 0.4.10 0 conda-forge postgresql 13.1 h6303168_2 conda-forge proj 8.0.0 h277dcde_0 conda-forge psycopg2 2.8.6 py39h3811e60_2 conda-forge pthread-stubs 0.4 h36c2ea0_1001 conda-forge pycparser 2.20 pyh9f0ad1d_2 conda-forge pygments 2.8.1 pyhd8ed1ab_0 conda-forge pyopenssl 20.0.1 pyhd8ed1ab_0 conda-forge pyparsing 2.4.7 pyh9f0ad1d_0 conda-forge pyproj 3.0.1 py39h0776cc1_1 conda-forge pyqt 5.12.3 py39hf3d152e_7 conda-forge pyqt-impl 5.12.3 py39h0fcd23e_7 conda-forge pyqt5-sip 4.19.18 py39he80948d_7 conda-forge pyqtchart 5.12 py39h0fcd23e_7 conda-forge pyqtwebengine 5.12.1 py39h0fcd23e_7 conda-forge pyqtwebkit 5.212 py39h20c28b1_1 conda-forge pysocks 1.7.1 py39hf3d152e_3 conda-forge python 3.9.2 hffdb5ce_0_cpython conda-forge python-dateutil 2.8.1 py_0 conda-forge python_abi 3.9 1_cp39 conda-forge pytz 2021.1 pyhd8ed1ab_0 conda-forge pyyaml 5.4.1 py39h3811e60_0 conda-forge qca 2.2.1 h73816c6_3 conda-forge qgis 3.18.2 py39h9afd837_1 conda-forge qjson 0.9.0 h73816c6_1006 conda-forge qscintilla2 2.11.2 py39hf01f197_4 conda-forge qt 5.12.9 hda022c4_4 conda-forge qtkeychain 0.12.0 h2264404_0 conda-forge qtlocation 5.12.9 he1b5a44_0 conda-forge qtserialport 5.9.8 h73816c6_1 conda-forge qtwebkit 5.212 h8f65c2e_1 conda-forge qwt 6.1.6 h7ec6b3e_0 conda-forge qwtpolar 1.1.1 h73816c6_7 conda-forge readline 8.1 h46c0cb4_0 conda-forge requests 2.25.1 pyhd3deb0d_0 conda-forge retrying 1.3.3 py_2 conda-forge setuptools 49.6.0 py39hf3d152e_3 conda-forge six 1.15.0 pyh9f0ad1d_0 conda-forge sqlite 3.35.4 h74cdb3f_0 conda-forge tiledb 2.2.7 h91fcb0e_0 conda-forge tk 8.6.10 hed695b0_1 conda-forge tzcode 2021a h7f98852_1 conda-forge tzdata 2021a he74cb21_0 conda-forge urllib3 1.26.4 pyhd8ed1ab_0 conda-forge wheel 0.36.2 pyhd3deb0d_0 conda-forge xerces-c 3.2.3 h9d8b166_2 conda-forge xorg-kbproto 1.0.7 h14c3975_1002 conda-forge xorg-libice 1.0.10 h516909a_0 conda-forge xorg-libsm 1.2.3 hd9c2040_1000 conda-forge xorg-libx11 1.7.0 h36c2ea0_0 conda-forge xorg-libxau 1.0.9 h14c3975_0 conda-forge xorg-libxdmcp 1.1.3 h516909a_0 conda-forge xorg-libxext 1.3.4 h7f98852_1 conda-forge xorg-libxrender 0.9.10 h7f98852_1003 conda-forge xorg-renderproto 0.11.1 h14c3975_1002 conda-forge xorg-xextproto 7.3.0 h14c3975_1002 conda-forge xorg-xproto 7.0.31 h14c3975_1007 conda-forge xz 5.2.5 h516909a_1 conda-forge yaml 0.2.5 h516909a_0 conda-forge zlib 1.2.11 h516909a_1010 conda-forge zstd 1.4.9 ha95c52a_0 conda-forge ```

I am using a fresh MambaForge:

conda info ``` active environment : qgisnumba active env location : /home/danschef/mambaforge/envs/qgisnumba shell level : 4 user config file : /home/daniel/.condarc populated config files : /home/daniel/mambaforge/.condarc conda version : 4.10.0 conda-build version : not installed python version : 3.8.8.final.0 virtual packages : __linux=4.10.0=0 __glibc=2.24=0 __unix=0=0 __archspec=1=x86_64 base environment : /home/danschef/mambaforge (writable) conda av data dir : /home/danschef/mambaforge/etc/conda conda av metadata url : https://repo.anaconda.com/pkgs/main channel URLs : https://conda.anaconda.org/conda-forge/linux-64 https://conda.anaconda.org/conda-forge/noarch package cache : /home/danschef/mambaforge/pkgs /home/danschef/.conda/pkgs envs directories : /home/danschef/mambaforge/envs /home/danschef/.conda/envs platform : linux-64 user-agent : conda/4.10.0 requests/2.25.1 CPython/3.8.8 Linux/4.10.0-40-generic ubuntu/17.04 glibc/2.24 UID:GID : 1000:1000 netrc file : None offline mode : False ```
chrisburr commented 3 years ago

What GPU and driver do you have on the problematic machine?

danschef commented 3 years ago

Does the glxinfo output help?

glxinfo ``` name of display: localhost:18.0 display: localhost:18 screen: 0 direct rendering: Yes server glx vendor string: SGI server glx version string: 1.4 server glx extensions: GLX_ARB_multisample, GLX_EXT_import_context, GLX_EXT_visual_info, GLX_EXT_visual_rating, GLX_MESA_copy_sub_buffer, GLX_OML_swap_method, GLX_SGIS_multisample, GLX_SGIX_fbconfig, GLX_SGIX_pbuffer, GLX_SGI_make_current_read, GLX_SGI_swap_control client glx vendor string: Mesa Project and SGI client glx version string: 1.4 client glx extensions: GLX_ARB_context_flush_control, GLX_ARB_create_context, GLX_ARB_create_context_no_error, GLX_ARB_create_context_profile, GLX_ARB_create_context_robustness, GLX_ARB_fbconfig_float, GLX_ARB_framebuffer_sRGB, GLX_ARB_get_proc_address, GLX_ARB_multisample, GLX_EXT_buffer_age, GLX_EXT_create_context_es2_profile, GLX_EXT_create_context_es_profile, GLX_EXT_fbconfig_packed_float, GLX_EXT_framebuffer_sRGB, GLX_EXT_import_context, GLX_EXT_texture_from_pixmap, GLX_EXT_visual_info, GLX_EXT_visual_rating, GLX_INTEL_swap_event, GLX_MESA_copy_sub_buffer, GLX_MESA_multithread_makecurrent, GLX_MESA_query_renderer, GLX_MESA_swap_control, GLX_OML_swap_method, GLX_OML_sync_control, GLX_SGIS_multisample, GLX_SGIX_fbconfig, GLX_SGIX_pbuffer, GLX_SGIX_visual_select_group, GLX_SGI_make_current_read, GLX_SGI_swap_control, GLX_SGI_video_sync GLX version: 1.4 GLX extensions: GLX_ARB_get_proc_address, GLX_ARB_multisample, GLX_EXT_import_context, GLX_EXT_visual_info, GLX_EXT_visual_rating, GLX_MESA_copy_sub_buffer, GLX_MESA_query_renderer, GLX_OML_swap_method, GLX_SGIS_multisample, GLX_SGIX_fbconfig, GLX_SGIX_pbuffer, GLX_SGI_make_current_read Extended renderer info (GLX_MESA_query_renderer): Vendor: VMware, Inc. (0xffffffff) Device: llvmpipe (LLVM 10.0.0, 256 bits) (0xffffffff) Version: 20.0.8 Accelerated: no Video memory: 257919MB Unified memory: no Preferred profile: core (0x1) Max core profile version: 3.3 Max compat profile version: 3.1 Max GLES1 profile version: 1.1 Max GLES[23] profile version: 3.1 OpenGL vendor string: VMware, Inc. OpenGL renderer string: llvmpipe (LLVM 10.0.0, 256 bits) OpenGL version string: 3.1 Mesa 20.0.8 OpenGL shading language version string: 1.40 OpenGL context flags: (none) OpenGL extensions: GL_AMD_conservative_depth, GL_AMD_draw_buffers_blend, GL_AMD_multi_draw_indirect, GL_AMD_query_buffer_object, GL_AMD_seamless_cubemap_per_texture, GL_AMD_shader_stencil_export, GL_AMD_shader_trinary_minmax, GL_AMD_texture_texture4, GL_ANGLE_texture_compression_dxt3, GL_ANGLE_texture_compression_dxt5, GL_APPLE_packed_pixels, GL_ARB_ES2_compatibility, GL_ARB_ES3_compatibility, GL_ARB_arrays_of_arrays, GL_ARB_base_instance, GL_ARB_blend_func_extended, GL_ARB_buffer_storage, GL_ARB_clear_buffer_object, GL_ARB_clear_texture, GL_ARB_clip_control, GL_ARB_color_buffer_float, GL_ARB_compatibility, GL_ARB_compressed_texture_pixel_storage, GL_ARB_compute_shader, GL_ARB_conditional_render_inverted, GL_ARB_conservative_depth, GL_ARB_copy_buffer, GL_ARB_copy_image, GL_ARB_cull_distance, GL_ARB_debug_output, GL_ARB_depth_buffer_float, GL_ARB_depth_clamp, GL_ARB_depth_texture, GL_ARB_derivative_control, GL_ARB_direct_state_access, GL_ARB_draw_buffers, GL_ARB_draw_buffers_blend, GL_ARB_draw_elements_base_vertex, GL_ARB_draw_indirect, GL_ARB_draw_instanced, GL_ARB_enhanced_layouts, GL_ARB_explicit_attrib_location, GL_ARB_explicit_uniform_location, GL_ARB_fragment_coord_conventions, GL_ARB_fragment_layer_viewport, GL_ARB_fragment_program, GL_ARB_fragment_program_shadow, GL_ARB_fragment_shader, GL_ARB_framebuffer_no_attachments, GL_ARB_framebuffer_object, GL_ARB_framebuffer_sRGB, GL_ARB_get_program_binary, GL_ARB_get_texture_sub_image, GL_ARB_half_float_pixel, GL_ARB_half_float_vertex, GL_ARB_indirect_parameters, GL_ARB_instanced_arrays, GL_ARB_internalformat_query, GL_ARB_internalformat_query2, GL_ARB_invalidate_subdata, GL_ARB_map_buffer_alignment, GL_ARB_map_buffer_range, GL_ARB_multi_bind, GL_ARB_multi_draw_indirect, GL_ARB_multisample, GL_ARB_multitexture, GL_ARB_occlusion_query, GL_ARB_occlusion_query2, GL_ARB_parallel_shader_compile, GL_ARB_pipeline_statistics_query, GL_ARB_pixel_buffer_object, GL_ARB_point_parameters, GL_ARB_point_sprite, GL_ARB_polygon_offset_clamp, GL_ARB_program_interface_query, GL_ARB_provoking_vertex, GL_ARB_query_buffer_object, GL_ARB_robustness, GL_ARB_sampler_objects, GL_ARB_seamless_cube_map, GL_ARB_seamless_cubemap_per_texture, GL_ARB_separate_shader_objects, GL_ARB_shader_atomic_counter_ops, GL_ARB_shader_atomic_counters, GL_ARB_shader_bit_encoding, GL_ARB_shader_draw_parameters, GL_ARB_shader_group_vote, GL_ARB_shader_image_load_store, GL_ARB_shader_image_size, GL_ARB_shader_objects, GL_ARB_shader_stencil_export, GL_ARB_shader_storage_buffer_object, GL_ARB_shader_subroutine, GL_ARB_shader_texture_lod, GL_ARB_shading_language_100, GL_ARB_shading_language_420pack, GL_ARB_shading_language_include, GL_ARB_shading_language_packing, GL_ARB_shadow, GL_ARB_stencil_texturing, GL_ARB_sync, GL_ARB_texture_border_clamp, GL_ARB_texture_buffer_object, GL_ARB_texture_buffer_object_rgb32, GL_ARB_texture_buffer_range, GL_ARB_texture_compression, GL_ARB_texture_compression_bptc, GL_ARB_texture_compression_rgtc, GL_ARB_texture_cube_map, GL_ARB_texture_cube_map_array, GL_ARB_texture_env_add, GL_ARB_texture_env_combine, GL_ARB_texture_env_crossbar, GL_ARB_texture_env_dot3, GL_ARB_texture_float, GL_ARB_texture_gather, GL_ARB_texture_mirror_clamp_to_edge, GL_ARB_texture_mirrored_repeat, GL_ARB_texture_multisample, GL_ARB_texture_non_power_of_two, GL_ARB_texture_query_levels, GL_ARB_texture_query_lod, GL_ARB_texture_rectangle, GL_ARB_texture_rg, GL_ARB_texture_rgb10_a2ui, GL_ARB_texture_stencil8, GL_ARB_texture_storage, GL_ARB_texture_storage_multisample, GL_ARB_texture_swizzle, GL_ARB_texture_view, GL_ARB_timer_query, GL_ARB_transform_feedback2, GL_ARB_transform_feedback3, GL_ARB_transform_feedback_instanced, GL_ARB_transform_feedback_overflow_query, GL_ARB_transpose_matrix, GL_ARB_uniform_buffer_object, GL_ARB_vertex_array_bgra, GL_ARB_vertex_array_object, GL_ARB_vertex_attrib_binding, GL_ARB_vertex_buffer_object, GL_ARB_vertex_program, GL_ARB_vertex_shader, GL_ARB_vertex_type_10f_11f_11f_rev, GL_ARB_vertex_type_2_10_10_10_rev, GL_ARB_viewport_array, GL_ARB_window_pos, GL_ATI_blend_equation_separate, GL_ATI_draw_buffers, GL_ATI_fragment_shader, GL_ATI_separate_stencil, GL_ATI_texture_compression_3dc, GL_ATI_texture_env_combine3, GL_ATI_texture_float, GL_ATI_texture_mirror_once, GL_EXT_EGL_image_storage, GL_EXT_EGL_sync, GL_EXT_abgr, GL_EXT_bgra, GL_EXT_blend_color, GL_EXT_blend_equation_separate, GL_EXT_blend_func_separate, GL_EXT_blend_minmax, GL_EXT_blend_subtract, GL_EXT_compiled_vertex_array, GL_EXT_copy_texture, GL_EXT_direct_state_access, GL_EXT_draw_buffers2, GL_EXT_draw_instanced, GL_EXT_draw_range_elements, GL_EXT_fog_coord, GL_EXT_framebuffer_blit, GL_EXT_framebuffer_multisample, GL_EXT_framebuffer_multisample_blit_scaled, GL_EXT_framebuffer_object, GL_EXT_framebuffer_sRGB, GL_EXT_gpu_program_parameters, GL_EXT_gpu_shader4, GL_EXT_multi_draw_arrays, GL_EXT_packed_depth_stencil, GL_EXT_packed_float, GL_EXT_packed_pixels, GL_EXT_pixel_buffer_object, GL_EXT_point_parameters, GL_EXT_polygon_offset_clamp, GL_EXT_provoking_vertex, GL_EXT_rescale_normal, GL_EXT_secondary_color, GL_EXT_separate_specular_color, GL_EXT_shader_integer_mix, GL_EXT_shadow_funcs, GL_EXT_stencil_two_side, GL_EXT_stencil_wrap, GL_EXT_subtexture, GL_EXT_texture, GL_EXT_texture3D, GL_EXT_texture_array, GL_EXT_texture_buffer_object, GL_EXT_texture_compression_dxt1, GL_EXT_texture_compression_latc, GL_EXT_texture_compression_rgtc, GL_EXT_texture_compression_s3tc, GL_EXT_texture_cube_map, GL_EXT_texture_edge_clamp, GL_EXT_texture_env_add, GL_EXT_texture_env_combine, GL_EXT_texture_env_dot3, GL_EXT_texture_integer, GL_EXT_texture_lod_bias, GL_EXT_texture_mirror_clamp, GL_EXT_texture_object, GL_EXT_texture_rectangle, GL_EXT_texture_sRGB, GL_EXT_texture_sRGB_R8, GL_EXT_texture_sRGB_decode, GL_EXT_texture_shared_exponent, GL_EXT_texture_snorm, GL_EXT_texture_swizzle, GL_EXT_timer_query, GL_EXT_transform_feedback, GL_EXT_vertex_array, GL_EXT_vertex_array_bgra, GL_IBM_multimode_draw_arrays, GL_IBM_rasterpos_clip, GL_IBM_texture_mirrored_repeat, GL_INGR_blend_func_separate, GL_KHR_context_flush_control, GL_KHR_debug, GL_KHR_no_error, GL_KHR_parallel_shader_compile, GL_KHR_texture_compression_astc_ldr, GL_KHR_texture_compression_astc_sliced_3d, GL_MESA_pack_invert, GL_MESA_shader_integer_functions, GL_MESA_texture_signed_rgba, GL_MESA_window_pos, GL_MESA_ycbcr_texture, GL_NV_blend_square, GL_NV_conditional_render, GL_NV_depth_clamp, GL_NV_fog_distance, GL_NV_light_max_exponent, GL_NV_packed_depth_stencil, GL_NV_primitive_restart, GL_NV_texgen_reflection, GL_NV_texture_env_combine4, GL_NV_texture_rectangle, GL_OES_EGL_image, GL_OES_read_format, GL_S3_s3tc, GL_SGIS_generate_mipmap, GL_SGIS_texture_border_clamp, GL_SGIS_texture_edge_clamp, GL_SGIS_texture_lod, GL_SUN_multi_draw_arrays 3 GLX Visuals visual x bf lv rg d st colorbuffer sr ax dp st accumbuffer ms cav id dep cl sp sz l ci b ro r g b a F gb bf th cl r g b a ns b eat ---------------------------------------------------------------------------- 0x083 24 tc 0 32 0 r . . 8 8 8 8 . . 0 0 0 0 0 0 0 0 0 None 0x086 24 tc 0 32 0 r . . 8 8 8 8 . . 0 24 8 0 0 0 0 0 0 None 0x087 24 tc 0 32 0 r . . 8 8 8 8 . . 0 16 0 0 0 0 0 0 0 None 4 GLXFBConfigs: visual x bf lv rg d st colorbuffer sr ax dp st accumbuffer ms cav id dep cl sp sz l ci b ro r g b a F gb bf th cl r g b a ns b eat ---------------------------------------------------------------------------- 0x042 24 tc 0 32 0 r . . 8 8 8 8 . . 0 0 0 0 0 0 0 0 0 None 0x045 24 tc 0 32 0 r . . 8 8 8 8 . . 0 24 8 0 0 0 0 0 0 None 0x047 24 tc 0 32 0 r . . 8 8 8 8 . . 0 16 0 0 0 0 0 0 0 None 0x049 0 tc 0 32 0 r y . 10 10 10 2 . . 0 24 8 0 0 0 0 0 0 None ```
chrisburr commented 3 years ago

Yup, that was very helpful. Did you try running with export LIBGL_ALWAYS_INDIRECT=1?

danschef commented 3 years ago

Wow, that works!

chrisburr commented 3 years ago

Excellent! In that case it is the mesa bug I that I linked earlier. Upgrading mesa should fix it but you probably don't want to do that unless you're upgrading the entire VM for other reasons. 😄

There is not much that can be done from conda-forge's perspective unfortunately so I'll close this.

danschef commented 3 years ago

I updated all packages on that server and got no error anymore. Thanks a lot for all for your efforts, that was really helpful!

I did: sudo apt update # 170 packages upgradable sudo apt upgrade sudo apt update # all packages up-to-date sudo add-apt-repository ppa:kisak/kisak-mesa sudo apt update # 14 packages upgradable sudo apt upgrade sudo apt update # all packages up-to-date sudo apt install mesa # E: Unable to locate package mesa

I am not sure why mesa can´t be located in the last step, however everything seems to work fine now.

Ofloo commented 1 year ago

rolled back to the amdgpu-install and now clinfo works again so both intel gpu and radeon gpu are working with opencl and obs works.

https://pastebin.com/uF3WzuTv