Open Keno opened 9 years ago
Interesting. I think this has no chance of working with static-libasan, so we should either remove DEEPBIND or complain about it or both. It works with -shared-libasan with clang but not with gcc, because gcc does not add DT_NEEDED for libasan to shared libraries.
On Mon, Oct 12, 2015 at 9:30 AM, Evgeniy Stepanov notifications@github.com wrote:
Interesting. I think this has no chance of working with static-libasan, so we should either remove DEEPBIND or complain
I'd say we need to complain and exit. (We may of course, have a flag to guard the logic here, but that's on overkill)
about it or both. It works with -shared-libasan with clang but not with gcc, because gcc does not add DT_NEEDED for libasan to shared libraries.
— Reply to this email directly or view it on GitHub https://github.com/google/sanitizers/issues/611#issuecomment-147453237.
Does this boil down to detecting RTLD_DEEPBIND in dlopen interceptor?
I'd guess so.
Hi! I have an environment in which I am using an external SDK. This SDK consist of a shared library I link with, and which makes use of dlopen to open other libX.so with RTLD_DEEPBIND flag. I'm not interested at all on sanitize the SDK, I only want to sanitize my own code.
How should I proceed? I'm confused about the different -shared-libasan/-static-libasan options.
As far as I understand, compiling with -static-libasan (GCC6) should inject all the sanitizer code in my binary and should have no effects on non-sanitized SDK, but I get an error on SDK trying to dlopen the .so:
Couldn't load libXXX.so: System error code(-2): libXXX.so: cannot open shared object file: No such file or directory
Using LD_LIBRARY_PATH to allow the SDK finding its libXXX.so I get the famous _int_free SEGV ASAN error. Seems like the SDK is redefining any alloc/free function, so there is a mismatch on symbol resolution (SDK vs Asan).
Looks like linking with -fsanitize=address changes the order of symbols lookup/LD_PATH. I would need to isolate the SDK so it remains using libc or its own implementation (I don't know how internally works and I don't have access to source code neither).
I want this behavior https://github.com/google/sanitizers/issues/871#issuecomment-337896857, but I'm not able to make the SDK run when my binary is using asan.
I am open to use blacklist, suppression list, attribute((no_sanitize("address"))) or any other option, but I would like to understand the problem.
Any hints? Thanks
Frankly, your best choice is to contact the vendor for the SDK and request that they support asan in whatever environment this SDK is being used. We've seen cases in the past when a closed-source SDK did something extremely asan-hostile and there is very little we could do there.
If you can provide a small reproducer, where you replace the SDK with a tiny mock library, please file a separate bug and we'll take a look (but no promises, such problems are often hard to solve)
On Mon, Oct 12, 2015 at 9:30 AM, Evgeniy Stepanov notifications@github.com wrote: I'd say we need to complain and exit. (We may of course, have a flag to guard the logic here, but that's on overkill)
Isn't it a good idea to add such flag? In code with a lot of modules this could help a lot as one wouldn't have to recompile some of them.
I am getting this error:
==7741==You are trying to dlopen a /usr/share/icedtea-web/lib/haswell/IcedTeaPlugin.so shared library with RTLD_DEEPBIND flag which is incompatibe with sanitizer runtime (see https://github.com/google/sanitizers/issues/611 for details). If you want to run /usr/share/icedtea-web/lib/haswell/IcedTeaPlugin.so library under sanitizers please remove RTLD_DEEPBIND from dlopen flags.
Interesting. I think this has no chance of working with static-libasan, so we should either remove DEEPBIND or complain about it or both. It works with -shared-libasan with clang but not with gcc, because gcc does not add DT_NEEDED for libasan to shared libraries.
Which version of clang could make -shared-libasan and DEEPBIND together?
I have tried clang 11 and 12 with -shared-libsan
or -shared-libasan
but still got error in runtime:
You are trying to dlopen a /usr/lib64/libssl.so shared library with RTLD_DEEPBIND flag which is incompatible with sanitizer runtime
Can I use [[no_sanitize("address")]]
to solve it?
Thanks
=====
btw: The problem I face, is some close source third library who depend on the other library which is not the same version on my host, so, denied the DEEPBIND is a problem for me. (for example loading both OpenSSL 1.0 and 1.1)
I have to use dlopen(RTLD_LOCAL|RTLD_DEEPBIND) in my code
, and dlmopen is also have a problem that some code not running on that namespace.
Maybe using blacklist is better than denied RTLD_DEEPBIND? Just warn and let user to add the relevant files into blacklist.
The other solution is to allow the RTLD_DEEPBIND to .so dynamic library which build with ASAN, so we can implement some shims easily. (such as https://github.com/yugr/Implib.so, https://github.com/dezgeg/libcapsule and other libGL shims)
Note: asan from gcc6 was ok with DEEPBIND, it started to fail from gcc8 (or 7?). Has anyone retried with gcc11 ?
tried, gcc-11.2.0 same problem, if a third-party library uses RTLD_DEEPBIND to load a shared library building with TSAN will now make the application report an error and refuse to start .
would it be possible to accept the fact that the code in the shared library can't be checked? In my case any application that links with this closed sources library can't use sanitizers anymore... It would be nice if I could still check my own code?
Same behavior with gcc11.3. +1 for @janwilmans proposal.
I encountered an interesting false-positive. When dlopen(RTLD_DEEPBIND)
a shared library which doesn't exist, Address Sanitizer will also die.
I encountered an interesting false-positive. When dlopen(RTLD_DEEPBIND) a shared library which doesn't exist, Address Sanitizer will also die.
ASan add this check in dlopen interceptor, if flags contain RTLD_DEEPBIND then ASan will report and die.
if (flag & RTLD_DEEPBIND) {
Report(
"You are trying to dlopen a %s shared library with RTLD_DEEPBIND flag"
" which is incompatible with sanitizer runtime "
"(see https://github.com/google/sanitizers/issues/611 for details"
"). If you want to run %s library under sanitizers please remove "
"RTLD_DEEPBIND from dlopen flags.\n",
filename, filename);
Die();
}
https://github.com/llvm/llvm-project/commit/76630d43f6081c657ac5be2406061d43334045dd added this check and it was well intentioned, however, if the library doesn't exist, there is no problem and the sanitizer still dies. Also the loaded library might never be actually be used (or its use is intentionally avoided, like we do) to allow you to still check your own code.
I encountered an interesting false-positive. When dlopen(RTLD_DEEPBIND) a shared library which doesn't exist, Address Sanitizer will also die.
ASan add this check in dlopen interceptor, if flags contain RTLD_DEEPBIND then ASan will report and die.
if (flag & RTLD_DEEPBIND) { Report( "You are trying to dlopen a %s shared library with RTLD_DEEPBIND flag" " which is incompatible with sanitizer runtime " "(see https://github.com/google/sanitizers/issues/611 for details" "). If you want to run %s library under sanitizers please remove " "RTLD_DEEPBIND from dlopen flags.\n", filename, filename); Die(); }
Yes, if the shared library at filename
doesn't exist, I think ASan needn't report and die.
The message was added because people we getting super weird false positives.
I think the message is very helpful, the Die()
, I have removed locally and I just know to that I can expect weirdness if I need this message. Maybe we should just add, ASAN can continue but please don't report issues if you see this message and be aware you can experience false positives from RTLD_DEEPBIND -loaded dynamic libraries.
Frankly, your best choice is to contact the vendor for the SDK and request that they support asan in whatever environment this SDK is being used. We've seen cases in the past when a closed-source SDK did something extremely asan-hostile and there is very little we could do there.
In practice this is very hard, some vendors aren't even aware the are doing this themselves, they load libraries, that load libraries etc. We should have some way to allow the user to sanitize their own code at least...
please remove RTLD_DEEPBIND from dlopen flags
It's intercepting the dlopen()
call already, so could asan provide a way to strip out the RTLD_DEEPBIND
flag before calling the real dlopen()
?
To follow the error message's suggestion and remove the flag it seems one currently has to rebuild the code which uses the flag. In the case I've just hit the problematic dlopen()
call is in the PHP interpreter, so that means rebuilding all of PHP in order to use asan on an externally built PHP module. Looking at PHP's code it already avoids using RTLD_DEEPBIND
automatically if PHP itself is built with asan or memsan, so if I could tell asan to strip out the flag it ought to work:
# if defined(RTLD_GROUP) && defined(RTLD_WORLD) && defined(RTLD_PARENT)
# define DL_LOAD(libname) dlopen(libname, PHP_RTLD_MODE | RTLD_GLOBAL | RTLD_GROUP | RTLD_WORLD | RTLD_PARENT)
# elif defined(RTLD_DEEPBIND) && !defined(__SANITIZE_ADDRESS__) && !__has_feature(memory_sanitizer)
# define DL_LOAD(libname) dlopen(libname, PHP_RTLD_MODE | RTLD_GLOBAL | RTLD_DEEPBIND)
# else
# define DL_LOAD(libname) dlopen(libname, PHP_RTLD_MODE | RTLD_GLOBAL)
# endif
@ojwb RTLD_DEEPBIND is there for a reason, you cannot just remove it and expect it to work ok. Specifically RTLD_DEEPBIND causes symbol lookups to happen in the loaded library and if you remove it, the symbols will be looked up in one of its parents, which is functionally not the same.
as a workaround solution:
colordiff --suppress-common-lines -U16 <(objdump -Mintel -dC /usr/lib/x86_64-linux-gnu/libasan.so.8.0.0) <(objdump -Mintel -dC /tmp/libasan.so.8.0.0)
0000000000060270 <__interceptor_dlopen.part.0>:
60270: 55 push rbp
60271: 48 89 e5 mov rbp,rsp
60274: 41 57 push r15
60276: 41 56 push r14
60278: 41 55 push r13
6027a: 41 54 push r12
6027c: 41 89 f4 mov r12d,esi
6027f: 53 push rbx
60280: 48 89 fb mov rbx,rdi
60283: 48 81 ec 28 08 00 00 sub rsp,0x828
6028a: 48 8d 05 0b 83 11 00 lea rax,[rip+0x11830b] # 17859c <__asan::asan_inited>
60291: 8b 30 mov esi,DWORD PTR [rax]
60293: 85 f6 test esi,esi
60295: 0f 84 95 01 00 00 je 60430 <__interceptor_dlopen.part.0+0x1c0>
6029b: 48 85 db test rbx,rbx
6029e: 74 10 je 602b0 <__interceptor_dlopen.part.0+0x40>
602a0: 48 8d 05 19 a0 1a 00 lea rax,[rip+0x1aa019] # 20a2c0 <__sanitizer::common_flags_dont_use>
602a7: 80 b8 f5 00 00 00 00 cmp BYTE PTR [rax+0xf5],0x0
602ae: 75 50 jne 60300 <__interceptor_dlopen.part.0+0x90>
602b0: 48 8d 05 29 4b 11 00 lea rax,[rip+0x114b29] # 174de0 <__asan::asan_flags_dont_use_directly>
602b7: 80 78 59 00 cmp BYTE PTR [rax+0x59],0x0
602bb: 0f 85 5f 01 00 00 jne 60420 <__interceptor_dlopen.part.0+0x1b0>
- 602c1: 44 89 e6 mov esi,r12d
+ 602c1: 31 f6 xor esi,esi
+ 602c3: 90 nop
602c4: 48 89 df mov rdi,rbx
602c7: e8 14 40 09 00 call f42e0 <__sanitizer::CheckNoDeepBind(char const*, int)>
602cc: 44 89 e6 mov esi,r12d
602cf: 48 89 df mov rdi,rbx
602d2: ff 15 98 4f 11 00 call QWORD PTR [rip+0x114f98] # 175270 <__interception::real_dlopen>
602d8: 48 89 c3 mov rbx,rax
602db: e8 40 2e 0a 00 call 103120 <__sanitizer::Symbolizer::GetOrInit()>
602e0: 48 89 c7 mov rdi,rax
602e3: e8 78 1f 0a 00 call 102260 <__sanitizer::Symbolizer::InvalidateModuleList()>
602e8: 48 8d 65 d8 lea rsp,[rbp-0x28]
602ec: 48 89 d8 mov rax,rbx
602ef: 5b pop rbx
602f0: 41 5c pop r12
602f2: 41 5d pop r13
602f4: 41 5e pop r14
602f6: 41 5f pop r15
602f8: 5d pop rbp
602f9: c3 ret
Please keep in mind, that this patch could break calls to free()
in some rare cases, when called from library which was loaded by dlopen()
I just found out that if an application opens a shared library (both built with address sanitizer) using RTLD_DEEPBIND, address sanitizer will fail to intercept the functions called from the so opened shared library. In my particular case, this manifested itself as a strdup which, though originally skipping any interceptors and going into libc, would eventually call asan's allocator. However the same did not happen for free (since it went straight into glibc's free), so it failed to intercept free (or rather there was an allocator mismatch). I am not familiar enough with glibc to say whether this can be fixed or not, but perhaps we could drop the RTLD_DEEPBIND flag in our dlopen interceptor and issue a warning to the user.