Sage should not blindly link to locally installed `libntl.a` (not compiled with -fPIC by default)

maxale commented 4 months ago

Environment

- **OS**: Ubuntu 22.04.4 LTS
- **Sage Version**: 10.4beta9

Steps To Reproduce

I had a locally installed NTL compiled from sources with default options. While Sage tries to link libntl.a, it fails with a few complaints like

[spkg-install] /usr/bin/ld: ///usr/local/lib/libntl.a(ZZX.o): relocation R_X86_64_TPOFF32 against `_ZZN3NTL8PlainSqrEPNS_2ZZEPKS0_lE36_ntl_hidden_variable_tls_local_ptr_t' can not be used when making a shared object; recompile with -fPIC

Config log

Relevant part from config.log:

## ---------------------------------------------------- ##
## Checking whether SageMath should install SPKG ntl... ##
## ---------------------------------------------------- ##
configure:23110: checking whether any of gmp gcc is installed as or will be installed as SPKG
configure:23120: result: no
configure:23123: checking for NTL/ZZ.h
configure:23123: g++ -std=gnu++11 -c    -Dlinux   -I/usr/include conftest.cpp >&5
configure:23123: $? = 0
configure:23123: result: yes
configure:23131: checking whether we can link a program using NTL
configure:23148: g++ -std=gnu++11 -o conftest    -Dlinux   -I/usr/include  conftest.cpp -lcurl -lcliquer -lcddgmp -lbz2 -lglpk -lgmp -lm  -lntl >&5
configure:23148: $? = 0
configure:23150: result: yes
configure:23161: checking NTL version >= 10.3
configure:23190: g++ -std=gnu++11 -o conftest    -Dlinux   -I/usr/include  conftest.cpp -lcurl -lcliquer -lcddgmp -lbz2 -lglpk -lgmp -lm  -lntl >&5
configure:23190: $? = 0
configure:23190: ./conftest
11.5.1
configure:23190: $? = 0
configure:23193: result: yes
configure:23219: will use system package and not install SPKG ntl
configure:23255: checking absolute name of <NTL/ZZ.h>
configure:23261: checking for NTL/ZZ.h
configure:23261: result: yes
configure:23282: result: ///usr/local/include/NTL/ZZ.h
configure:23361: $PKG_CONFIG --exists --print-errors "ntl"
Package ntl was not found in the pkg-config search path.
Perhaps you should add the directory containing `ntl.pc'
to the PKG_CONFIG_PATH environment variable
No package 'ntl' found
configure:23364: $? = 1
configure:23394: $PKG_CONFIG --exists --print-errors "ntl"
Package ntl was not found in the pkg-config search path.
Perhaps you should add the directory containing `ntl.pc'
to the PKG_CONFIG_PATH environment variable
No package 'ntl' found
configure:23397: $? = 1

Package logs

No response

Additional Information

To overcome the issue I had to go to NTL sources, manually add -fPIC compiler option to its makefile, recompile and reinstall my local libntl.a. However, I think this issue should have an automated solution - e.g., better testing of possibility for linking.

Also, while Sage says configure:23219: will use system package and not install SPKG ntl, it uses not the system-provided package (which would be /usr/lib/x86_64-linux-gnu/libntl.a) but a locally-installed one (/usr/local/lib/libntl.a) instead.

Checklist

[X] I have searched the existing issues for a bug report that matches the one I want to file, without success.
[X] I have read the documentation and troubleshoot guide

mkoeppe commented 4 months ago

PR welcome.

dimpase commented 4 months ago

Also, while Sage says configure:23219: will use system package and not install SPKG ntl, it uses not the system-provided package (which would be /usr/lib/x86_64-linux-gnu/libntl.a) but a locally-installed one (/usr/local/lib/libntl.a) instead.

that's what we understand by "system". Something that is found by the compiler/linker's default settings. In particular /usr/local is a legitimate location in this sense. It's also pretty much standard on macOS (with Homebrew) and various *BSD systems. As well, Conda's packages are supported by this mechanism (one can't really mix Conda and non-Conda, though).

dimpase commented 4 months ago

We do run a test program using NTL in ./configure, but it does even link NTLs library, it only uses its headers. It's easy to add a test which does something like shown in the beginning of https://libntl.org/doc/tour-ex1.html

I'll do a PR to add this.

mkoeppe commented 4 months ago

The problem with the existing autoconf macros for link and run testing is that they do NOT enforce use of shared libraries. So these link and run tests will pass even when only static (non-reloc) libs are present. But in our use of the libraries, we do enforce the use of shared libraries.

maxale commented 2 months ago

There may be another issue caused by linking Sage to libntl.a installed in the system. Please see my comment at https://github.com/sagemath/sage/issues/38376#issuecomment-2332448665

sagemath / sage