search

conda-forge / llvmlite-feedstock

A conda-smithy repository for llvmlite.
BSD 3-Clause "New" or "Revised" License
0 stars 21 forks source link

Unnecessary libllvm dependency for numba through llvmlite #84

Open ziw-liu opened 8 months ago

ziw-liu commented 8 months ago

Solution to issue cannot be found in the documentation.

Issue

The conda-forge recipe for numba hard-pins libllvm to 14.0, which breaks OpenGL-based software when the llvmpipe renderer is used for mesa on Linux.

https://github.com/conda-forge/llvmlite-feedstock/blob/ca6e60523541905df2482e8f2ff893dce40fede8/recipe/meta.yaml#L32

However, the upstream documentation specifically explains that only llvmlite is used and any LLVM installation will be ignored:

You do not need to have LLVM installed to use Numba (in fact, Numba will ignore all LLVM versions installed on the system) as the required components are bundled into the llvmlite wheel.

Also, installing from upstream channel will not install libllvm, and it works with llvmpipe:

conda install -c numba numba

Installed packages

# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
bzip2                     1.0.8                hd590300_5    conda-forge
ca-certificates           2024.2.2             hbcca054_0    conda-forge
ld_impl_linux-64          2.40                 h41732ed_0    conda-forge
libblas                   3.9.0           21_linux64_openblas    conda-forge
libcblas                  3.9.0           21_linux64_openblas    conda-forge
libexpat                  2.5.0                hcb278e6_1    conda-forge
libffi                    3.4.4                h6a678d5_0  
libgcc-ng                 13.2.0               h807b86a_5    conda-forge
libgfortran-ng            13.2.0               h69a702a_5    conda-forge
libgfortran5              13.2.0               ha4646dd_5    conda-forge
libgomp                   13.2.0               h807b86a_5    conda-forge
liblapack                 3.9.0           21_linux64_openblas    conda-forge
libllvm14                 14.0.6               hcd5def8_4    conda-forge
libnsl                    2.0.1                hd590300_0    conda-forge
libopenblas               0.3.26          pthreads_h413a1c8_0    conda-forge
libsqlite                 3.44.2               h2797004_0    conda-forge
libstdcxx-ng              13.2.0               h7e041cc_5    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libxcrypt                 4.4.36               hd590300_1    conda-forge
libzlib                   1.2.13               hd590300_5    conda-forge
llvmlite                  0.42.0          py311ha6695c7_1    conda-forge
ncurses                   6.4                  h59595ed_2    conda-forge
numba                     0.59.0          py311h96b013e_1    conda-forge
numpy                     1.26.4          py311h64a7726_0    conda-forge
openssl                   3.2.1                hd590300_0    conda-forge
pip                       24.0               pyhd8ed1ab_0    conda-forge
python                    3.11.7          hab00c5b_1_cpython    conda-forge
python_abi                3.11                    4_cp311    conda-forge
readline                  8.2                  h8228510_1    conda-forge
setuptools                69.0.3             pyhd8ed1ab_0    conda-forge
tk                        8.6.13          noxft_h4845f30_101    conda-forge
tzdata                    2024a                h0c530f3_0    conda-forge
wheel                     0.42.0             pyhd8ed1ab_0    conda-forge
xz                        5.4.5                h5eee18b_0

Environment info

active environment : numba
    active env location : ...
            shell level : 2
       user config file : ...
 populated config files : ...
          conda version : 4.12.0
    conda-build version : 3.21.8
         python version : 3.9.12.final.0
       virtual packages : __cuda=12.3=0
                          __linux=4.18.0=0
                          __glibc=2.28=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : ...
      conda av data dir : ...
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : ...
       envs directories : ...
               platform : linux-64
             user-agent : conda/4.12.0 requests/2.27.1 CPython/3.9.12 Linux/4.18.0-477.27.1.el8_8.x86_64 rocky/8.8 glibc/2.28
                UID:GID : 5464:5464
             netrc file : None
           offline mode : False
jakirkham commented 8 months ago

We dynamically link to LLVM. Hence the library is a required dependency for us

However the runtime library is named libllvm14, which includes the version in the name

Another package could depend on a different LLVM runtime library (for example libllvm15) and both could be installed without clobbering each other

A little unclear what the other package is that you are using, but maybe it would be worth following up with them in a new issue to switch to the versioned LLVM library

ziw-liu commented 8 months ago

The conflicted 'package' is not a conda package, it's the system's graphics driver, the llvmpipe OpenGL renderer from Mesa. It will just try to use whatever LLVM it can find (it will be conda-installed libllvm14 in this case) to JIT-compile shaders. This is not what an unprivileged user can hope to control, and very difficult to downgrade on a HPC system.

Is there a reason to ship a dynamically linked LLVM? Since Numba's official builds seem to be running fine on all the platforms (including arm, powerpc etc) by bundling LLVM into llvmlite.

ziw-liu commented 8 months ago

Here is the backtrace:

#0  0x0000153a43e09c9f _ZL15AddNodeIDCustomRN4llvm16FoldingSetNodeIDEPKNS_6SDNodeE (libLLVM-14.so)
#1  0x0000153a43e09d54 _ZN4llvm10FoldingSetINS_6SDNodeEE10NodeEqualsEPKNS_14FoldingSetBaseEPNS3_4NodeERKNS_16FoldingSetNodeIDEjRS8_ (libLLVM-14.so)
#2  0x00001539deedc0ee _ZN4llvm14FoldingSetBase19FindNodeOrInsertPosERKNS_16FoldingSetNodeIDERPvRKNS0_14FoldingSetInfoE (libLLVM-15.so)
#3  0x00001539df98c692 _ZN4llvm12SelectionDAG19FindNodeOrInsertPosERKNS_16FoldingSetNodeIDERKNS_5SDLocERPv (libLLVM-15.so)
#4  0x00001539df9bd866 _ZN4llvm12SelectionDAG7getNodeEjRKNS_5SDLocENS_3EVTENS_7SDValueES5_NS_11SDNodeFlagsE (libLLVM-15.so)
#5  0x00001539df96129b _ZN4llvm19SelectionDAGBuilder18visitShuffleVectorERKNS_4UserE (libLLVM-15.so)
#6  0x00001539df986a13 _ZN4llvm19SelectionDAGBuilder5visitERKNS_11InstructionE (libLLVM-15.so)
#7  0x00001539df9f2624 _ZN4llvm16SelectionDAGISel16SelectBasicBlockENS_14ilist_iteratorINS_12ilist_detail12node_optionsINS_11InstructionELb0ELb0EvEELb0ELb1EEES6_Rb (libLLVM-15.so)
#8  0x00001539df9f3b12 _ZN4llvm16SelectionDAGISel20SelectAllBasicBlocksERKNS_8FunctionE (libLLVM-15.so)
#9  0x00001539df9f629a _ZN4llvm16SelectionDAGISel20runOnMachineFunctionERNS_15MachineFunctionE.part.0 (libLLVM-15.so)
#10 0x00001539e2690c93 _ZN12_GLOBAL__N_115X86DAGToDAGISel20runOnMachineFunctionERN4llvm15MachineFunctionE (libLLVM-15.so)
#11 0x00001539df443644 _ZN4llvm19MachineFunctionPass13runOnFunctionERNS_8FunctionE.part.0 (libLLVM-15.so)
#12 0x00001539df154031 _ZN4llvm13FPPassManager13runOnFunctionERNS_8FunctionE (libLLVM-15.so)
#13 0x00001539df15416c _ZN4llvm13FPPassManager11runOnModuleERNS_6ModuleE (libLLVM-15.so)
#14 0x00001539df154aef _ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE (libLLVM-15.so)
#15 0x00001539e11e87e0 _ZN4llvm5MCJIT10emitObjectEPNS_6ModuleE (libLLVM-15.so)
#16 0x00001539e11e8e9e _ZN4llvm5MCJIT21generateCodeForModuleEPNS_6ModuleE (libLLVM-15.so)
#17 0x00001539e11e4b11 _ZN4llvm5MCJIT14finalizeObjectEv (libLLVM-15.so)
#18 0x00001539e116289f LLVMGetPointerToGlobal (libLLVM-15.so)
#19 0x00001539e618eeeb llvmpipe_update_fs (swrast_dri.so)
#20 0x00001539e617f7a8 llvmpipe_update_derived (swrast_dri.so)
#21 0x00001539e615c8d8 llvmpipe_draw_vbo (swrast_dri.so)
#22 0x00001539e5cf33f5 _mesa_draw_arrays.part.11 (swrast_dri.so)

The Mesa driver gets confused between llvm 14 and 15, and segfaults.

esc commented 8 months ago

Numba requires a specific LLVM to work correctly. Currently for 0.59 this is LLVM 14. Numba and llvmlite are then built differently by distributors:

The Numba and llvmlite form PyPi and from the Numba channel (-c numba) on anaconda.org have LLVM bundled into llvmlite (static linking).

The Numba and llvmlite from conda-forge and from the defaults channel on anaconda.org do depend on a libllvm package (dynamic linking).

These setups are very unlikely to change, so I would recommend using a Numba and llvmlite combo from a distributor such that you can work around the issues you are having.

Alternatively, perhaps there is an environment variable that can be set for llvmpipe to point that package to a suitable LLVM?

ziw-liu commented 8 months ago

Alternatively, perhaps there is an environment variable that can be set for llvmpipe to point that package to a suitable LLVM?

I cannot find a way to point llvmpipe to a specific LLVM at runtime other than setting a global LD_LIBRARY_PATH, which will also affect other packages.

For now the best workaround may be using the numba channel instead.

esc commented 8 months ago

Alternatively, perhaps there is an environment variable that can be set for llvmpipe to point that package to a suitable LLVM?

I cannot find a way to point llvmpipe to a specific LLVM at runtime other than setting a global LD_LIBRARY_PATH, which will also affect other packages.

For now the best workaround may be using the numba channel instead.

Yes. I hope that works for you, do let us know how it goes!

It may be worth adding an LLVM_CONFIG feature to llvmpipe or asking for that feature to be implemented on that project! Shouldn't be too hard of an addition.