stuartarchibald
commented
9 months ago We build LLVM and llvmlite from source, and our LLVM is configured to build both static and dynamic libraries. I noticed that llvmlite was getting dynamically linked against libLLVM-14.so. My sense is that llvmlite intends to be statically linked to libLLVM per https://llvmlite.readthedocs.io/en/latest/faqs.html , so I think the build system should enforce it.
For the make-based build, adding
--link-staticto thellvm-config --libscall fixes it, but for the CMake build, I'm not exactly sure what to do. The lib/cmake/llvm/ I get seems to list both static and dynamic libraries and that's how my libllvmlite.so is getting linked if I set LLVMLITE_USE_CMAKE=1, and I am not quite familiar enough with CMake to know why.If you would like, I am happy to send you a pull request for the makefile change alone if you think that's useful by itself (but it's also a one-line change).
On the other hand, if both static and dynamic linking is supported but your wheels statically link LLVM because shipping the dynamic libraries in a wheel is both useless and painful, that makes perfect sense and I am happy to send you a PR to clarify the FAQ. Dynamic linking was working fine for us, and I think libLLVM is good enough at symbol versioning that the concern in that FAQ about multiple versions is not so relevant these days - you should be able to safely import both llvmlite and something else that uses e.g. libLLVM-15.so into the same process.
Thanks for raising this @geofft, it will be discussed at the maintainer's meeting next week. Both static and dynamic linking are supported, though it would appear from the above that the CMake build may need some investigation/adjustment. Most "distributors" e.g. conda packagers, linux distributions etc. are using dynamic linking, whereas wheels use static (as noted above, it's hard to do dynamic linking in wheels etc). Numba conda channel builds use static linking too, this is so that the maintainers can be absolutely sure about the linkage chain and provision of symbols (making sure that the Numba stack is isolated from external influence where possible makes it easier to debug). Historically, collisions across multiple LLVM versions have been a problem and this could arise from e.g. very old LLVM versions in use in surprising places through to various build/configuration options, though it may indeed be less of an issue now.
(The issue template wants me to confirm that this affects the latest release and to attach a reproducer. It does affect the latest release as well as main, but a reproducer is going to be a little complicated... let me know if you want me to write a shell script to build LLVM from source and demonstrate the problem :) )
I think omitting this is entirely reasonable!
geofft
seibert
We build LLVM and llvmlite from source, and our LLVM is configured to build both static and dynamic libraries. I noticed that llvmlite was getting dynamically linked against libLLVM-14.so. My sense is that llvmlite intends to be statically linked to libLLVM per https://llvmlite.readthedocs.io/en/latest/faqs.html , so I think the build system should enforce it.
For the make-based build, adding
--link-staticto thellvm-config --libscall fixes it, but for the CMake build, I'm not exactly sure what to do. The lib/cmake/llvm/ I get seems to list both static and dynamic libraries and that's how my libllvmlite.so is getting linked if I set LLVMLITE_USE_CMAKE=1, and I am not quite familiar enough with CMake to know why.If you would like, I am happy to send you a pull request for the makefile change alone if you think that's useful by itself (but it's also a one-line change).
On the other hand, if both static and dynamic linking is supported but your wheels statically link LLVM because shipping the dynamic libraries in a wheel is both useless and painful, that makes perfect sense and I am happy to send you a PR to clarify the FAQ. Dynamic linking was working fine for us, and I think libLLVM is good enough at symbol versioning that the concern in that FAQ about multiple versions is not so relevant these days - you should be able to safely import both llvmlite and something else that uses e.g. libLLVM-15.so into the same process.
(The issue template wants me to confirm that this affects the latest release and to attach a reproducer. It does affect the latest release as well as main, but a reproducer is going to be a little complicated... let me know if you want me to write a shell script to build LLVM from source and demonstrate the problem :) )