LeakSanitizer has encountered a fatal error

[mustafaahmedhussien]

I am facing with one my binaries the same issue of https://github.com/google/sanitizers/issues/764

But I am running on an ubuntu 18 machine with no docker.

==4608==AddressSanitizer: libc interceptors initialized ||[0x10007fff8000, 0x7fffffffffff]|| HighMem || ||[0x02008fff7000, 0x10007fff7fff]|| HighShadow || ||[0x00008fff7000, 0x02008fff6fff]|| ShadowGap || ||[0x00007fff8000, 0x00008fff6fff]|| LowShadow || ||[0x000000000000, 0x00007fff7fff]|| LowMem || MemToShadow(shadow): 0x00008fff7000 0x000091ff6dff 0x004091ff6e00 0x02008fff6fff redzone=16 max_redzone=2048 quarantine_size_mb=256M thread_local_quarantine_size_kb=1024K malloc_context_size=30 SHADOW_SCALE: 3 SHADOW_GRANULARITY: 8 SHADOW_OFFSET: 0x7fff8000 ==4608==Installed the sigaction for signal 11 ==4608==Installed the sigaction for signal 7 ==4608==Installed the sigaction for signal 8 ==4608==T0: stack [0x7ffe62265000,0x7ffe62a65000) size 0x800000; local=0x7ffe62a62b58 ==4608==AddressSanitizer Init done ==4608==T1: stack [0x7f7e8c007000,0x7f7e8c405a80) size 0x3fea80; local=0x7f7e8c405988 ==4608==T2: stack [0x7f7e8bc06000,0x7f7e8c004a80) size 0x3fea80; local=0x7f7e8c004988 ==4608==T3: stack [0x7f7e8b805000,0x7f7e8bc03a80) size 0x3fea80; local=0x7f7e8bc03988 ==4608==T4: stack [0x7f7e8b3f6000,0x7f7e8b7f4a80) size 0x3fea80; local=0x7f7e8b7f4988 ==4608==T6: stack [0x7f7e8abd8000,0x7f7e8afd6a80) size 0x3fea80; local=0x7f7e8afd6988 ==4608==T5: stack [0x7f7e8afe7000,0x7f7e8b3e5a80) size 0x3fea80; local=0x7f7e8b3e5988 ==4608==T8: stack [0x7f7e8a3c0000,0x7f7e8a7bea80) size 0x3fea80; local=0x7f7e8a7be988 ==4608==T7: stack [0x7f7e8a7c9000,0x7f7e8abc7a80) size 0x3fea80; local=0x7f7e8abc7988 ==4608==T10: stack [0x7f7e89b9a000,0x7f7e89f98a80) size 0x3fea80; local=0x7f7e89f98988 ==4608==T9: stack [0x7f7e89fa3000,0x7f7e8a3a1a80) size 0x3fea80; local=0x7f7e8a3a1988 ==4608==T11: stack [0x7f7e8976d000,0x7f7e89b6ba80) size 0x3fea80; local=0x7f7e89b6b988 ==4608==T12: stack [0x7f7e88f3e000,0x7f7e8973ca80) size 0x7fea80; local=0x7f7e8973c988 ==4608==T12 TSDDtor ==4608==T12 exited ==4621==Processing thread 4608. ==4621==Stack at 0x7ffe62265000-0x7ffe62a65000 (SP = 0x7ffe62a628a8). ==4621==TLS at 0x7f7eb69b9000-0x7f7eb69ba580. ==4621==Processing thread 4609. ==4621==Stack at 0x7f7e8c007000-0x7f7e8c405a80 (SP = 0x7f7e8c405908). ==4621==TLS at 0x7f7e8c405a80-0x7f7e8c407000. ==4621==DTLS 7 at 0x1f80000c20000010-0x2280000b20000112. Tracer caught signal 11: addr=0x0 pc=0x5092b8 sp=0x7f7eaf532d10 ==4608==LeakSanitizer has encountered a fatal error. ==4608==HINT: For debugging, try setting environment variable LSAN_OPTIONS=verbosity=1:log_threads=1 ==4608==HINT: LeakSanitizer does not work under ptrace (strace, gdb, etc)

[kcc]

Is that just your binary or any binary on that machine?

[mustafaahmedhussien]

It is only for this library. ptrace is set to 0.

[kcc]

I don't think we can help w/o a reproducer.

[stsquad]

I have a reproducer when running:

qemu-aarch64 ./tests/tcg/aarch64-linux-user/semihosting

On the current QEMU master built with clang and --enable-sanitizers

./aarch64-linux-user/qemu-aarch64 ./tests/tcg/aarch64-linux-user/semihosting
Hello WorldTracer caught signal 11: addr=0x56067e44c000 pc=0x56067aa53a10 sp=0x7f4b79dfdc20
==6039==LeakSanitizer has encountered a fatal error.
==6039==HINT: For debugging, try setting environment variable LSAN_OPTIONS=verbosity=1:log_threads=1
==6039==HINT: LeakSanitizer does not work under ptrace (strace, gdb, etc)

If I enable debug I get:

env LSAN_OPTIONS=verbosity=1:log_threads=1 ./aarch64-linux-user/qemu-aarch64 ./tests/tcg/aarch64-linux-user/semihosting
==7276==AddressSanitizer: libc interceptors initialized
|| `[0x10007fff8000, 0x7fffffffffff]` || HighMem    ||
|| `[0x02008fff7000, 0x10007fff7fff]` || HighShadow ||
|| `[0x00008fff7000, 0x02008fff6fff]` || ShadowGap  ||
|| `[0x00007fff8000, 0x00008fff6fff]` || LowShadow  ||
|| `[0x000000000000, 0x00007fff7fff]` || LowMem     ||
MemToShadow(shadow): 0x00008fff7000 0x000091ff6dff 0x004091ff6e00 0x02008fff6fff
redzone=16
max_redzone=2048
quarantine_size_mb=256M
thread_local_quarantine_size_kb=1024K
malloc_context_size=30
SHADOW_SCALE: 3
SHADOW_GRANULARITY: 8
SHADOW_OFFSET: 0x7fff8000
==7276==Installed the sigaction for signal 11
==7276==Installed the sigaction for signal 7
==7276==Installed the sigaction for signal 8
==7276==T0: stack [0x7fffb9cd2000,0x7fffba4d2000) size 0x800000; local=0x7fffba4cfa94
==7276==AddressSanitizer Init done
==7276==T1: stack [0x7f1ed4500000,0x7f1ed4cfeec0) size 0x7feec0; local=0x7f1ed4cfedb4
Hello WorldTracer caught signal 11: addr=0x5566fd444000 pc=0x5566f9a4ba10 sp=0x7f1ed3cfdc20
==7276==LeakSanitizer has encountered a fatal error.
==7276==HINT: For debugging, try setting environment variable LSAN_OPTIONS=verbosity=1:log_threads=1
==7276==HINT: LeakSanitizer does not work under ptrace (strace, gdb, etc)

I haven't found a suitable breakpoint to catch the seg or examine the code so it's hard to know where the PC is pointing.

[stsquad]

Using the gdb attach after crash trick:

(gdb) x/5i 0x556c17460a10
   0x556c17460a10 <_ZN6__lsan20ScanRangeForPointersEmmPN11__sanitizer18InternalMmapVectorImEEPKcNS_8ChunkTagE+176>:     mov    (%rbx),%r14
   0x556c17460a13 <_ZN6__lsan20ScanRangeForPointersEmmPN11__sanitizer18InternalMmapVectorImEEPKcNS_8ChunkTagE+179>:     lea    -0x4000(%r14),%rax
   0x556c17460a1a <_ZN6__lsan20ScanRangeForPointersEmmPN11__sanitizer18InternalMmapVectorImEEPKcNS_8ChunkTagE+186>:     cmp    %r13,%rax
   0x556c17460a1d <_ZN6__lsan20ScanRangeForPointersEmmPN11__sanitizer18InternalMmapVectorImEEPKcNS_8ChunkTagE+189>:     jae    0x556c17460b58 <_ZN6__lsan20ScanRangeForPointersEmmPN11__sanitizer18InternalMmapVectorImEEPKcNS_8ChunkTagE+504>
   0x556c17460a23 <_ZN6__lsan20ScanRangeForPointersEmmPN11__sanitizer18InternalMmapVectorImEEPKcNS_8ChunkTagE+195>:     mov    %r14,%rdi