[NSAN] numerical-sanitizer not available as shared library

[VinInn]

the numerical sanitizer is currently linked as whole-archive in the main program. -shared-libsan does not seem to have any effect. nsan symbols are left undefined when building a shared library that will fail to link if -Wl,-z,defs is used. This makes the numerical sanitizer unusable in any "plugin" to be dynamically loaded by an external framework.

[MaskRay]

In Clang, most static runtime files are only used when linking executables. https://maskray.me/blog/2023-01-08-all-about-sanitizer-interceptors#static-runtime

When linking a shared object, the static runtime is not used. The linker option -z defs (--no-undefined) may lead to undefined symbol errors.

% clang -fsanitize=address -shared -fpic a.c -Wl,-z,defs
ld.lld: error: undefined symbol: __asan_report_load8

To support an uninstrumented executable with instrumented DSOs, -fsanitize=numerical can add a shared runtime. Let me take a stab.

[VinInn]

It seems to work. It requires to explicitly add -shared-libsan -rpath ${CLANG_BASE}/lib/clang/19/lib/x86_64-unknown-linux-gnu/ when building the plugin though

[VinInn]

actually the library cannot be dynamically loaded

touch foo.cc
clang++ -O3 -ffast-math -g -Wall -shared -fPIC foo.cc -o libfoo.so -fsanitize=numerical  -Wl,-z,defs -shared-libsan -rpath  /afs/cern.ch/work/i/innocent/public/w5/lib/clang/20//lib/x86_64-unknown-linux-gnu
python3 -i
Python 3.6.8 (default, May 31 2023, 10:28:59)
[GCC 8.5.0 20210514 (Red Hat 8.5.0-18)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import *
>>> cdll.LoadLibrary("./libfoo.so")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python3.6/ctypes/__init__.py", line 421, in LoadLibrary
    return self._dlltype(name)
  File "/usr/lib64/python3.6/ctypes/__init__.py", line 343, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /afs/cern.ch/work/i/innocent/public/w5/lib/clang/20//lib/x86_64-unknown-linux-gnu/libclang_rt.nsan.so: cannot allocate memory in static TLS block

[VinInn]

Should I open a new issue for the "dlopen" failure?

[alexander-shaposhnikov]

cc: @vitalybuka

[alexander-shaposhnikov]

@VinInn - thanks for the report, one thing that I would check - I'd probably try to load other runtimes (e.g. msan, tsan) to see if they can be loaded successfully. This wouldn't immediately help with the issue but might be a useful data point.

[VinInn]

tsan has the same issue. memprof and ubsan are ok asan requires to be loaded as first library

python -i
>>> from ctypes import *
>>> dir = '/afs/cern.ch/work/i/innocent/public/w5/lib/clang/20//lib/x86_64-unknown-linux-gnu/'
>>> cdll.LoadLibrary(dir+'libclang_rt.nsan.so')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python2.7/ctypes/__init__.py", line 439, in LoadLibrary
    return self._dlltype(name)
  File "/usr/lib64/python2.7/ctypes/__init__.py", line 361, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /afs/cern.ch/work/i/innocent/public/w5/lib/clang/20//lib/x86_64-unknown-linux-gnu/libclang_rt.nsan.so: cannot allocate memory in static TLS block
>>> cdll.LoadLibrary(dir+'libclang_rt.tsan.so')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python2.7/ctypes/__init__.py", line 439, in LoadLibrary
    return self._dlltype(name)
  File "/usr/lib64/python2.7/ctypes/__init__.py", line 361, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /afs/cern.ch/work/i/innocent/public/w5/lib/clang/20//lib/x86_64-unknown-linux-gnu/libclang_rt.tsan.so: cannot allocate memory in static TLS block
>>> cdll.LoadLibrary(dir+'libclang_rt.memprof.so')
<CDLL '/afs/cern.ch/work/i/innocent/public/w5/lib/clang/20//lib/x86_64-unknown-linux-gnu/libclang_rt.memprof.so', handle 55cf57bc1a80 at 7f703408ebd0>
>>> cdll.LoadLibrary(dir+'libclang_rt.ubsan_minimal.so')
<CDLL '/afs/cern.ch/work/i/innocent/public/w5/lib/clang/20//lib/x86_64-unknown-linux-gnu/libclang_rt.ubsan_minimal.so', handle 55cf57ba5150 at 7f703409b110>
>>> cdll.LoadLibrary(dir+'libclang_rt.asan.so')
==3411047==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD.

[VinInn]

for tsan preloading "works" preloading nsan segfaults

[innocent@patatrack01]~% export LD_PRELOAD=/afs/cern.ch/work/i/innocent/public/w5/lib/clang/20//lib/x86_64-unknown-linux-gnu/libclang_rt.tsan.so
[innocent@patatrack01]~%  python -i
Python 2.7.18 (default, Dec 21 2022, 18:08:05)
[GCC 8.5.0 20210514 (Red Hat 8.5.0-17)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import *
>>> cdll.LoadLibrary('/afs/cern.ch/work/i/innocent/public/w5/lib/clang/20//lib/x86_64-unknown-linux-gnu/libclang_rt.tsan.so')
<CDLL '/afs/cern.ch/work/i/innocent/public/w5/lib/clang/20//lib/x86_64-unknown-linux-gnu/libclang_rt.tsan.so', handle 7f35e8347460 at 7f35e3db6110>
>>>
[innocent@patatrack01]~% export LD_PRELOAD=/afs/cern.ch/work/i/innocent/public/w5/lib/clang/20//lib/x86_64-unknown-linux-gnu/libclang_rt.nsan.so
[innocent@patatrack01]~%  python -i
zsh: segmentation fault (core dumped)  python -i