sanitizer dlopen interception clobbers RUNPATH

[fjricci]

Bugzilla Link	27790
Version	3.8
OS	Linux
CC	@compnerd,@eugenis,@fjricci,@kcc,@pawelsopensource,@vitalybuka

Extended Description

dlopen() uses the RUNPATH of the dso which calls it. This means that when a shared sanitizer dso intercepts dlopen, the RUNPATH of the sanitizer dso will be used instead of the RUNPATH of the dso being intercepted.

Repro steps:

clang-3.8 test.c -Wl,-rpath,'$ORIGIN' -Wl,--enable-new-dtags -ldl -o clean
clang-3.8 test.c -Wl,-rpath,'$ORIGIN' -Wl,--enable-new-dtags -ldl -fsanitize=address -o sanitized_static
clang-3.8 test.c -Wl,-rpath,'$ORIGIN' -Wl,--enable-new-dtags -ldl -fsanitize=address -shared-libasan -o sanitized_dynamic
clang-3.8 foo.c -shared -o foo.so

Note that executing clean and sanitized_static will succeed, while executing sanitized_dynamic will fail.

foo.c:

int foo = 0;

test.c:

#include <dlfcn.h>

int main() {
  if (dlopen("foo.so", RTLD_NOW)) {
    return 0;
  }

  return 1;
}

[vitalybuka]

So WONTFIX?

[llvmbot]

Sorry, only noticed this now (my Samsung address is inactive). I'm afraid I don't see how we could fix this easily and not intercepting dlopen is undesirable too. Perhaps just treat this as known issue?

[fjricci]

Yes. If asan is linked statically, the dlopen call will happen from within the original binary (or DSO), which will have the correct RUNPATH set. It's only a problem if dlopen is called from within the asan dso.

[kcc]

So, the problem only appears with -shared-libasan? Adding the Yuri and Max who have more experience with shared-libasan

[yshui]

this hurts when developing on NixOS. one workaround is setting a shellHook = "export LD_LIBRARY_PATH=/run/opengl-driver/lib", if for example you need libGL. but I sure hope this workaround is not needed.

can't sanitizer dlopen look at the source shared object and find its rpath?

[keith]

I think the original statement about this only applying when using -shared-libasan is incorrect. When using statically linked asan the same thing can happen when you have 1 shared library that loads another. The setup looks like this:

// main.cc
#include <dlfcn.h>
#include <iostream>

int main() {
  std::cout << "start main\n";
  ::dlopen("foo.so", RTLD_NOW);
  std::cout << dlerror() << "\n";
  std::cout << "end main\n";
}

// foo.cc
#include <cstdint>
#include <dlfcn.h>
#include <iostream>

__attribute__((constructor))
void loadbar() {
  std::cout << "start  foo\n";
  ::dlopen("bar.so", RTLD_NOW);
  std::cout << "end  foo\n";
}

// bar.cc
#include <iostream>

__attribute__((constructor))
void runbar() {
  std::cout << "loaded bar\n";
}

# Makefile
foo/foo.so: lib.h foo.cc
    clang++ -shared foo.cc '-Wl,-rpath,$$ORIGIN/../bar' -o foo/foo.so -fsanitize=address

bar/bar.so: lib.h bar.cc
    clang++ -shared bar.cc -o bar/bar.so -fsanitize=address

main: main.cc lib.h foo/foo.so bar/bar.so
    clang++ main.cc '-Wl,-rpath,$$ORIGIN/foo' -o main -fsanitize=address

clean:
    rm -f main bar/bar.so foo/foo.so

In this case even with asan linked statically:

% ldd main
        linux-vdso.so.1 (0x0000f3e552cf3000)
        libstdc++.so.6 => /lib/aarch64-linux-gnu/libstdc++.so.6 (0x0000f3e551f00000)
        libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000f3e551e60000)
        libresolv.so.2 => /lib/aarch64-linux-gnu/libresolv.so.2 (0x0000f3e551e30000)
        libgcc_s.so.1 => /lib/aarch64-linux-gnu/libgcc_s.so.1 (0x0000f3e551e00000)
        libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000f3e551c50000)
        /lib/ld-linux-aarch64.so.1 (0x0000f3e552cba000)

It will fail to load bar at runtime, and when you run with LD_DEBUG=libs you can see it's using the RUNPATH from main:

    237995:     find library=bar.so [0]; searching
    237995:      search path=/home/ubuntu/repro-dlopen-issue/foo                (RUNPATH from file ./main)
    237995:       trying file=/home/ubuntu/repro-dlopen-issue/foo/bar.so
    237995:      search cache=/etc/ld.so.cache
    237995:      search path=(SNIP)         (system search path)
    SNIP
end  foo
bar.so: cannot open shared object file: No such file or directory
end main

Here's this project for testing: repro-dlopen-issue.zip