compiling with asan generates wrong DWARF

GoogleCodeExporter commented 9 years ago

What steps will reproduce the problem?
1. Compile the attached file with:
clang -fsanitize=address asan.cc -g -o hello -O0
2. Inspect the dwarf, looking at the locals of main() argc and argv

What is the expected output? What do you see instead?
It turns out that the DWARF is emitting location info for the locals:
0x0000005f:     TAG_subprogram [5] *
                 AT_name( "main" )
                 AT_decl_file( "/private/tmp/asan/src.cc" )
                 AT_decl_line( 5 )
                 AT_type( {0x00000058} ( int ) )
                 AT_external( 0x01 )
                 AT_low_pc( 0x0000000100000b90 )
                 AT_high_pc( 0x0000000100000e87 )
                 AT_frame_base( rbp )

0x0000007d:         TAG_formal_parameter [3]  
                     AT_name( "argc" )
                     AT_decl_file( "/private/tmp/asan/src.cc" )
                     AT_decl_line( 5 )
                     AT_type( {0x00000058} ( int ) )
                     AT_location( 0x00000038
                        0x0000000100000b90 - 0x0000000100000ca4: rdi
                        0x0000000100000ca4 - 0x0000000100000cbe: rsp+48, deref )

0x0000008c:         TAG_formal_parameter [3]  
                     AT_name( "argv" )
                     AT_decl_file( "/private/tmp/asan/src.cc" )
                     AT_decl_line( 5 )
                     AT_type( {0x000000a8} ( char** ) )
                     AT_location( 0x00000070
                        0x0000000100000b90 - 0x0000000100000ca9: r8
                        0x0000000100000ca9 - 0x0000000100000cbe: rsp+40, deref )

but the ranges are too small to cover the entire function space, e.g. for "argv"

                     AT_location( 0x00000070
                        0x0000000100000b90 - 0x0000000100000ca9: r8
                        0x0000000100000ca9 - 0x0000000100000cbe: rsp+40, deref )

So when debugging in LLDB, we stop at 
(lldb) reg read pc
     rip = 0x0000000100000d84  hello`main + 500 at asan.cc:6
and the function PC space is:
                 AT_low_pc( 0x0000000100000b90 )
                 AT_high_pc( 0x0000000100000e87 )
but while we are validly inside main(), the DWARF fails to vend a location for 
the, existing and available, locals
If one manually follows the directions given by DWARF:
                        0x0000000100000ca4 - 0x0000000100000cbe: rsp+48, deref )
(lldb) expr *((int*)$rsp+48)
(int) $5 = 4 <— this is assuming one launches the inferior in LLDB with, say, 
"run 1 2 3"

the right value comes out

It looks like it might simply be a matter of extending the location to cover 
the whole address space

This reproduces for me on OS X with ToT clang/asan

Original issue reported on code.google.com by granata....@gmail.com on 21 Oct 2013 at 5:40

Attachments:

asan.cc

GoogleCodeExporter commented 9 years ago

Original comment by konstant...@gmail.com on 21 Oct 2013 at 6:39

GoogleCodeExporter commented 9 years ago

Original comment by samso...@google.com on 29 Oct 2013 at 5:10

Changed state: Accepted

GoogleCodeExporter commented 9 years ago

I've bisected this to LLVM r185966. Investigating.

Original comment by samso...@google.com on 29 Oct 2013 at 6:41

GoogleCodeExporter commented 9 years ago

It seems that revision just exposed the problem described here: 
http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20131014/191274.html
 (and introduced as early as r128327). Long story short,
LLVM CodeGen sometimes incorrectly assumes that expressions which define the 
location of a user variable in memory are only valid until the end of a basic 
block, while instead they are vaild until the end of the function. This bug 
hurts plain Clang -O0 debug info as well, but only occasionally. However, all 
ASan full debug info is broken because of that.

Original comment by samso...@google.com on 30 Oct 2013 at 12:08

GoogleCodeExporter commented 9 years ago

It would be great to get this fixed, it makes debugging ASAN binaries 
impossible which is important of course when analyzing how a memory error 
occurred.

Is there a workaround possible?

Original comment by mric...@googlemail.com on 23 May 2014 at 5:24

GoogleCodeExporter commented 9 years ago

I have a patch that fixes this issue for me, and the necessary changes are 
currently being reviewed. I can you send you a patch tomorrow, so that you can 
verify it helps your debugging. For now I can suggest no reasonable workaround 
(except for debugging the binary w/o ASan, or figuring out where the locals are 
stored by reading assembly, not nice).

Original comment by samso...@google.com on 23 May 2014 at 5:41

GoogleCodeExporter commented 9 years ago

Yes, please send me the patch, I will try it out also.

2014-05-23 7:41 GMT+02:00 <address-sanitizer@googlecode.com>:

Original comment by mric...@googlemail.com on 23 May 2014 at 2:57

GoogleCodeExporter commented 9 years ago

Here's the patch - you'll need to apply it to LLVM trunk andr re-build Clang.

Original comment by samso...@google.com on 23 May 2014 at 11:29

Attachments:

zdiff.i235

GoogleCodeExporter commented 9 years ago

[deleted comment]

GoogleCodeExporter commented 9 years ago

Any updates here?

Original comment by bent.moz...@gmail.com on 27 Aug 2014 at 5:24

GoogleCodeExporter commented 9 years ago

Sorry, I was sure I've updated this bug. I've submitted r210492 a while ago 
that should significantly improve the situation with debug info for local 
variables under ASan. There are still things to fix, but locals should now 
mostly work (at least with -O0). I'm dropping a priority for this bug. Please 
report if you see / don't see improvements on your side.

Original comment by samso...@google.com on 27 Aug 2014 at 7:40

Added labels: Priority-Low
Removed labels: Priority-Medium

GoogleCodeExporter commented 9 years ago

Hi,

with clang 3.5 this is still broken for simple cases. I tested on SLES 11 
x86_64. Please see the attached example. A debugger session is pasted in 
build.sh, and you can also run build.sh to generate the binary "gtest_asan" 
which reproduces the problem. Both a local variable and the "this" pointer get 
optimized out and can not be displayed in the debugger.

One precondition seems to be that an external function call must be there, 
though I was only able to reproduce it with the gtest framework, but not with a 
simple extern function.

Interestingly, the debug information for the local variable is there if 
compiled with -O1 ("this" pointer is still optimized out).

I did not test this on trunk yet.

Best regards,
Martin Richtarsky

Original comment by mric...@googlemail.com on 25 Nov 2014 at 1:19

Attachments:

debuginfo.tar.gz

GoogleCodeExporter commented 9 years ago

Original comment by samso...@google.com on 26 Nov 2014 at 11:05

Added labels: Priority-High
Removed labels: Priority-Low

GoogleCodeExporter commented 9 years ago

Hi,

just wanted to ask how other people debug ASan reports, is there a workaround 
for this bug here? I have found quite a few cases where debugging would have 
been helpful, in some cases it is only possible to reason about the error with 
proper debugging info. Are there debuggers which work with clang+ASan 3.5? Or 
is everybody just fixing the errors with the callstacks?

Thanks and Best regards,
Martin

Original comment by mric...@googlemail.com on 29 Dec 2014 at 7:36

GoogleCodeExporter commented 9 years ago

I'm not aware of existing debuggers that workaround this issue with ASan debug 
info quality. BTW, if you have tip-of-trunk Clang, you can check if the 
following compile flag improves the debug info quality:
  clang++ -fsanitize=address -mllvm -asan-stack-dynamic-alloca=1 my/file.cc
It fixes test case in 
https://code.google.com/p/address-sanitizer/issues/detail?id=235#c12 for me, 
and I hope to make this option the default after some testing.

Original comment by samso...@google.com on 29 Dec 2014 at 6:54

GoogleCodeExporter commented 9 years ago

Thanks, "-mllvm -asan-stack-dynamic-alloca=1" solves the problem for me aswell. 
Debug info is looking good in general.

It also solves a problem I was having with dynamic alloca() code (same as 
https://code.google.com/p/address-sanitizer/issues/detail?id=138 I guess)

I will test further with the flag. Let me know if I can help in any way to make 
this the default.

Best regards,
Martin

Original comment by mric...@googlemail.com on 30 Dec 2014 at 12:48

GoogleCodeExporter commented 9 years ago

Thanks for trying this. Please report any problems you encounter. If all goes 
well, I hope to flip the default early next year.

Original comment by samso...@chromium.org on 30 Dec 2014 at 7:15

GoogleCodeExporter commented 9 years ago

I've flipped the default flag value in LLVM r228336

Original comment by samso...@google.com on 5 Feb 2015 at 9:02

Changed state: Fixed

gopalshankar / address-sanitizer

compiling with asan generates wrong DWARF #235