search

libntl / ntl

258 stars 51 forks source link

Linking issue when using non system GMP #30

Open antoine-morvan opened 3 months ago

antoine-morvan commented 3 months ago

Hello,

I encountered some issues when building NTL along with a non-system GMP, but with GMP installed on the system. There are 2 problems related :

A. setup3 rule does not propagate $(GMP_OPT_LIB) to binary

setup3:
    $(LINK) $(GMP_OPT_INCDIR) -o gen_gmp_aux gen_gmp_aux.cpp $(GMP_OPT_LIBDIR) $(GMP_OPT_LIB) $(LDLIBS)
    ./gen_gmp_aux > ../include/NTL/gmp_aux.h

The -L<path to gmp /lib> is given to the linker, but the next line executing the program will not be aware (when linking to dynamic gmp). This leads to the common issue : 

GMP version check (6.2.1/6.0.0)
*** version number mismatch: inconsistency between gmp.h and libgmp

This can be mitigated with

  1. The LD_LIBRARY_PATH is set with the same gmp folder, as advised in other issues relating the problem
  2. $(GMP_OPT_LIB) actually also encodes an rpath

Point one is not always possible, e.g., when installing GMP and NTL in a prefix along with a new libreadline setup. Using LD_LIBRARY_PATH would most likely have your bash segfault.

Point 2 is the solution used by package managers like nix or spack, to avoid the segfault issue described above.

B. The configure script does not read LDFLAGS from the environment

My workaround is to use LDFLAGS with rpath, so that it does not affect by current $SHELL. However the configure script does not read LDFLAGS from the environment, and I had to explicitly pass it to the script :

export LDFLAGS="XXX"
./configure # ignores LDFLAGS

./configure LDFLAGS="XXX" # properly sets ld flags

Had few hours trying to figure this out ^^'

The point A seems kinda tricky to address to me, not sure if there is a best practice. The c++ code states that this is inspired from MPFR : https://github.com/libntl/ntl/blob/91acd5b3a7df709c0d8bf88a99a24bc340dc34f7/src/gen_gmp_aux.cpp#L113-L126 I did not, however, enounter any similar issue when building MPFR. Maybe they have a new way for checking this ?

On the other hand, I believe having the configure script reading the LDFLAGS from the environnement is reasonable... although I just give a workaround here :)

Best.

victorshoup commented 3 months ago

It’s been a while since I thought about this issue. Whenever I’m installing, I usually just make sure LD_LIBRARY_PATH is set appropriately, usually just setting it once and for all in .bashrc or whatever. I admit this may be a limited solution, but can you describe in detail the use cases that would not be covered by this?

Thanks.

antoine-morvan commented 3 months ago

Well I don't have it anymore because I switched to rpath, but I had a situation where GMP was installed along some other libraries under an arbitrary prefix. Exporting LD_LIBRARY_PATH actually tells ALL the binaries of the system to link dynamically against that prefix first. And one of my system binaries would segfault because not compatible with one of the libs (that would take precedence over the system, lib). I think it was bash, but it could be anything (and everything). You can easily imagine that happening with a GLIBC installed in the prefix: you do not want your system binaries to link against it. Many online doc/tuto do warn to be carefull with this, e.g. https://www.baeldung.com/linux/multiple-glibc :

Alternatively, we could also export the LD_LIBRARY_PATH to include the path to our glibc directory before running our application. However, any new child processes launched by the program should also use the newer glib as specified in this variable.

That's also related in nix/spack/etc. package managers issue trackers, motivating them to move to rpath instead of LD_LIBRARY_PATH. They have some examples with low level libraries like gettext, iconv, etc. producing unstable behavior.

I can try to build a reproducer again if needed and if I find some time ;)

antoine-morvan commented 3 months ago

Found the spack one : https://github.com/spack/spack/issues/3955

victorshoup commented 3 months ago

Does cmake handle this stuff better? If so, maybe it’s time to make the switch. I’ve been very hesitant about embracing cmake. On Jun 5, 2024, at 11:44 AM, Antoine Morvan @.***> wrote: Well I don't have it anymore because I switched to rpath, but I had a situation where GMP was installed along some other libraries under an arbitrary prefix. Exporting LD_LIBRARY_PATH actually tells ALL the binaries of the system to link dynamically against that prefix first. And one of my system binaries would segfault because not compatible with one of the libs (that would take precedence over the system, lib). I think it was bash, but it could be anything (and everything). You can easily imagine that happening with a GLIBC installed in the prefix: you do not want your system binaries to link against it. Many online doc/tuto do warn to be carefull with this, e.g. https://www.baeldung.com/linux/multiple-glibc :

Alternatively, we could also export the LD_LIBRARY_PATH to include the path to our glibc directory before running our application. However, any new child processes launched by the program should also use the newer glib as specified in this variable.

That's also related in nix/spack/etc. package managers issue trackers, motivating them to move to rpath instead of LD_LIBRARY_PATH. They have some examples with low level libraries like gettext, iconv, etc. producing unstable behavior. I can try to build a reproducer again if needed and if I find some time ;)

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

antoine-morvan commented 3 months ago

Maybe CMake comes with better modules regarding internal test binary/libs generation w.r.t. the LD_LIBRARY_PATH or ldflags. But I am no cmake expert (nor autotools actually). As a user I have no preference: both allow ugly things, and both have their own pro&cons.