On OS X, stacktraces are broken after a report is printed

[GoogleCodeExporter]

When the stacktraces are captured and printed by ASan itself, they are fine, 
but when the program has already printed the report (or is just printing it), 
capturing a stacktrace via other means is broken. "Other means" include OS X 
CrashReporter, debuggers or calling backtrace() within the program. For example 
calling backtrace() from a sanitizer_set_death_callback function prints a very 
truncated stacktrace:

    0   crashlog-stacktraces.c.tmp          0x000000010fb1c397 death_function + 471
    1   libclang_rt.asan_osx_dynamic.dylib  0x000000010fb72c1f _ZN11__sanitizer3DieEv + 15

The same can happen even within lldb (I'm seeing this for 32-bit x86):

    (lldb) bt
    * thread #1: tid = 0x9ec2d, 0x0000209a crashlog-stacktraces.c.tmp`death_function + 42 at crashlog-stacktraces.c:10, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
      * frame #0: 0x0000209a crashlog-stacktraces.c.tmp`death_function + 42 at crashlog-stacktraces.c:10
        frame #1: 0x0005eaf6 libclang_rt.asan_osx_dynamic.dylib`__sanitizer::Die() + 22
        frame #2: 0x00058136 libclang_rt.asan_osx_dynamic.dylib`__asan::ScopedInErrorReport::~ScopedInErrorReport() + 102
        frame #3: 0x00057fcf libclang_rt.asan_osx_dynamic.dylib`__asan::ScopedInErrorReport::~ScopedInErrorReport() + 15
        frame #4: 0x00057acc libclang_rt.asan_osx_dynamic.dylib`__asan_report_error + 4204
    (lldb) 

It's probably a combination of "noreturn" and -fomit-frame-pointer, that 
confuses the unwinders too much. This can be "fixed" if we don't use the 
-fomit-frame-pointer flag (now we use it for all sources in compiler-rt), but 
I'm curious if there is a less dramatic solution. If I remember correctly, 
-fomit-frame-pointer was important for performance reasons. Maybe forcing the 
frame pointer to be used in all noreturn functions could be enough?

Attaching a testcase that reproduces this (Darwin-specific).

Original issue reported on code.google.com by kuba.brecka@gmail.com on 21 Jan 2015 at 3:41

[GoogleCodeExporter]

// RUN: %clang_asan -O0 %s -o %t
// RUN: not %run %t 2>&1 | FileCheck %s

#include <execinfo.h>
#include <sanitizer/common_interface_defs.h>
#include <stdio.h>
#include <stdlib.h>

void death_function() {
  fprintf(stderr, "DEATH CALLBACK\n");

  void* callstack[128];
  int i, frames = backtrace(callstack, 128);
  char** strs = backtrace_symbols(callstack, frames);
  for (i = 0; i < frames; ++i) {
    fprintf(stderr, "%s\n", strs[i]);
  }
  free(strs);

  fprintf(stderr, "END OF BACKTRACE\n");
}

int fault_function() {
  char *x = (char*)malloc(10 * sizeof(char));
  free(x);
  return x[5];  // BOOM
}

int main() {
  __sanitizer_set_death_callback(death_function);
  fault_function();
  return 0;
}

// CHECK: {{.*ERROR: AddressSanitizer: heap-use-after-free on address}}
// CHECK: {{READ of size 1 at 0x.* thread T0}}
// CHECK: {{    #0 0x.* in fault_function}}

// CHECK: DEATH CALLBACK
// CHECK: death_function
// CHECK: fault_function
// CHECK: main
// CHECK: END OF BACKTRACE

Original comment by kuba.brecka@gmail.com on 21 Jan 2015 at 3:42

[GoogleCodeExporter]

Isn't unwind info sufficient for CrashReporter to unwind the stack? Note that 
ASan runtime is able to successfully unwind despite the missing stack traces.

Can we just pass the stack trace collected by ASan to CrashReporter?

Original comment by ramosian.glider@gmail.com on 21 Jan 2015 at 8:29

[GoogleCodeExporter]

Although CrashReporter has some private API to add information to a crashlog, 
that would only add additional information. The main stack trace is still taken 
by CrashReporter itself, and tools that work with crash logs will use this 
trace.

But I'm more surprised that LLDB is actually truncating the stack trace as 
well, which makes debugging terrible, and that also makes this look like 
something worth fixing in ASan itself.

It also seems to me that this could be done by forcing having a stackframe in 
__asan_report_error only, by using the ENABLE_FRAME_POINTER macro, which we're 
already using elsewhere (asan_mac.cc). What do you think?

Original comment by kuba.brecka@gmail.com on 21 Jan 2015 at 6:41

[GoogleCodeExporter]

Seems like a bug or limitation in LLDB. It's probably worth working around in 
ASan, but it would be good to get this reduced and filed upstream with LLDB too.

Original comment by rnk@google.com on 21 Jan 2015 at 6:50

[GoogleCodeExporter]

Submitted a proposed patch into http://reviews.llvm.org/D7103

Original comment by kuba.brecka@gmail.com on 21 Jan 2015 at 7:18

[GoogleCodeExporter]

So doesn't CrashReporter use unwind info rather than frame pointers to unwind 
the stack?
I'm ok with the proposed patch, but still curious why it's necessary.

Original comment by ramosian.glider@gmail.com on 22 Jan 2015 at 1:53

[GoogleCodeExporter]

CrashReporter and backtrace() currently unwind via eh_frame and/or stack 
frames.  That works fine for most code, including system libraries, because on 
OS X system libraries (usually) don't use -fomit-frame-pointer.  LLDB does know 
how to use the compact unwind info, but apparently for i386 there is a separate 
bug in LLDB, which I filed as rdar://problem/19570035.

Anyway, if it's okay, I'm going to submit the proposed workaround that forces a 
proper stack frame on __asan_report_error (http://reviews.llvm.org/D7103).

Original comment by kuba.brecka@gmail.com on 22 Jan 2015 at 11:30

[GoogleCodeExporter]

Committed the workaround in r226878.

Original comment by kuba.brecka@gmail.com on 22 Jan 2015 at 11:39

[GoogleCodeExporter]

The LLDB issue is fixed with r226889: 
http://lists.cs.uiuc.edu/pipermail/lldb-commits/Week-of-Mon-20150119/015056.html

Original comment by kuba.brecka@gmail.com on 23 Jan 2015 at 3:29

[GoogleCodeExporter]

Adding Project:AddressSanitizer as part of GitHub migration.

Original comment by ramosian.glider@gmail.com on 30 Jul 2015 at 9:06

• Added labels: ProjectAddressSanitizer