minidump_stackwalk: segmentation fault at linked_ptr_internal::join on linux x64

[GoogleCodeExporter]

What steps will reproduce the problem?
1.build breakpad on linux x64(Linux 10.1.1.101 3.5.7 #2 SMP Tue Jan 29 17:30:27 
CST 2013 x86_64 GNU/Linux).
2.the crash.dump in attachment is a crash dump file on Android ARM.
3.libcrash.so is the crash so, and symbols/ contains is symbol file converted 
by dump_syms.
4.run 
$ ./minidump_stackwalk crash.dump symbols > /dev/null 2>errorlog
[1]    4981 segmentation fault  ./minidump_stackwalk crash.dump symbols > 
/dev/null 2> errorlog
(gdb) bt
#0  0x00000000004110e3 in google_breakpad::RangeMap<unsigned long, 
google_breakpad::linked_ptr<google_breakpad::BasicSourceLineResolver::Function> 
>::RetrieveRange (this=0x26a0fd0, 
    address=@0x7fffd966f4f0, entry=0x7fffd966f4c0, entry_base=0x7fffd966f4e8, entry_size=0x7fffd966f4e0) at ./src/processor/linked_ptr.h:75
#1  0x00000000004113c5 in google_breakpad::RangeMap<unsigned long, 
google_breakpad::linked_ptr<google_breakpad::BasicSourceLineResolver::Function> 
>::RetrieveNearestRange (this=0x26a0fd0, 
    address=@0x7fffd966f4f0, entry=0x26a5858, entry_base=0x7fffd966f4e8, entry_size=0x7fffd966f4e0) at ./src/processor/range_map-inl.h:154
#2  0x000000000040d997 in 
google_breakpad::BasicSourceLineResolver::Module::LookupAddress 
(this=0x26a0f90, frame=0x7fffd966f670) at 
src/processor/basic_source_line_resolver.cc:203
#3  0x0000000000439517 in 
google_breakpad::SourceLineResolverBase::FillSourceLineInfo 
(this=0x7fffd966fd60, frame=0x7fffd966f670) at 
src/processor/source_line_resolver_base.cc:302
#4  0x000000000043c207 in 
google_breakpad::StackFrameSymbolizer::FillSourceLineInfo (this=0x26871f0, 
modules=<value optimized out>, system_info=0x7fffd966fce8, frame=0x7fffd966f670)
    at src/processor/stack_frame_symbolizer.cc:76
#5  0x000000000043c91a in 
google_breakpad::Stackwalker::InstructionAddressSeemsValid (this=0x26a08f0, 
address=<value optimized out>) at src/processor/stackwalker.cc:236
#6  0x000000000043ee51 in google_breakpad::StackwalkerARM::GetCallerByStackScan 
(this=0x26a08f0, frames=<value optimized out>) at 
./src/google_breakpad/processor/stackwalker.h:165
#7  0x000000000043ff17 in google_breakpad::StackwalkerARM::GetCallerFrame 
(this=0x26a08f0, stack=0x26a0c30) at src/processor/stackwalker_arm.cc:263
#8  0x000000000043cdaf in google_breakpad::Stackwalker::Walk (this=0x26a08f0, 
stack=0x26a0c30, modules_without_symbols=0x7fffd966fd20, 
modules_with_corrupt_symbols=0x7fffd966fd38)
    at src/processor/stackwalker.cc:162
#9  0x0000000000435b68 in google_breakpad::MinidumpProcessor::Process 
(this=0x7fffd966fdb0, dump=0x7fffd966fba0, process_state=0x7fffd966fc90) at 
src/processor/minidump_processor.cc:249
#10 0x00000000004369ee in google_breakpad::MinidumpProcessor::Process 
(this=0x7fffd966fdb0, minidump_file=@0x7fffd966fdd0, 
process_state=0x7fffd966fc90) at src/processor/minidump_processor.cc:310
#11 0x00000000004095b0 in main (argc=<value optimized out>, argv=<value 
optimized out>) at src/processor/minidump_stackwalk.cc:594

$ tail errorlog
...
2013-07-24 18:01:01: basic_code_modules.cc:88: INFO: No module at 0xbeb04468
2013-07-24 18:01:01: basic_code_modules.cc:88: INFO: No module at 0xbeb044b0
2013-07-24 18:01:01: basic_code_modules.cc:88: INFO: No module at 0xbeb043b8
2013-07-24 18:01:01: basic_code_modules.cc:88: INFO: No module at 0x41b1bdc4

Ooops, crashed after trying to find modules, remeber the address "0x41b1bdc4", 
later this string crashes the link list!

$ gdb --args ./minidump_stackwalk crash.dump symbols
(gdb) r
...
Program received signal SIGSEGV, Segmentation fault.
0x00000000004110e3 in google_breakpad::RangeMap<unsigned long, 
google_breakpad::linked_ptr<google_breakpad::BasicSourceLineResolver::Function> 
>::RetrieveRange (this=0x26a0fd0, 
    address=@0x7fffd966f4f0, entry=0x7fffd966f4c0, entry_base=0x7fffd966f4e8, entry_size=0x7fffd966f4e0) at ./src/processor/linked_ptr.h:75
75          while (p->next_ != ptr) p = p->next_;

/* src/processor/linked_ptr.h

  void join(linked_ptr_internal const* ptr) {
    linked_ptr_internal const* p = ptr;
    while (p->next_ != ptr) p = p->next_;
    p->next_ = this;
    next_ = ptr;
  }
*/

(gdb) display/i $pc // crashed at +147
1: x/i $pc
0x4110e3 
<_ZNK15google_breakpad8RangeMapImNS_10linked_ptrINS_23BasicSourceLineResolver8Fu
nctionEEEE13RetrieveRangeERKmPS4_PmS9_+147>:   mov    (%rax),%rax

(gdb) disassemble 
...
0x00000000004110d9 
<_ZNK15google_breakpad8RangeMapImNS_10linked_ptrINS_23BasicSourceLineResolver8Fu
nctionEEEE13RetrieveRangeERKmPS4_PmS9_+137>: lea    0x38(%rbx),%rdx
0x00000000004110dd 
<_ZNK15google_breakpad8RangeMapImNS_10linked_ptrINS_23BasicSourceLineResolver8Fu
nctionEEEE13RetrieveRangeERKmPS4_PmS9_+141>: mov    %rdx,%rax
0x00000000004110e0 
<_ZNK15google_breakpad8RangeMapImNS_10linked_ptrINS_23BasicSourceLineResolver8Fu
nctionEEEE13RetrieveRangeERKmPS4_PmS9_+144>: mov    %rax,%rcx
0x00000000004110e3 
<_ZNK15google_breakpad8RangeMapImNS_10linked_ptrINS_23BasicSourceLineResolver8Fu
nctionEEEE13RetrieveRangeERKmPS4_PmS9_+147>: mov    (%rax),%rax
0x00000000004110e6 
<_ZNK15google_breakpad8RangeMapImNS_10linked_ptrINS_23BasicSourceLineResolver8Fu
nctionEEEE13RetrieveRangeERKmPS4_PmS9_+150>: cmp    %rax,%rdx
0x00000000004110e9 
<_ZNK15google_breakpad8RangeMapImNS_10linked_ptrINS_23BasicSourceLineResolver8Fu
nctionEEEE13RetrieveRangeERKmPS4_PmS9_+153>: jne    0x4110e0 
<_ZNK15google_breakpad8RangeMapImNS_10linked_ptrINS_23BasicSourceLineResolver8Fu
nctionEEEE13RetrieveRangeERKmPS4_PmS9_+144>
...

// It is looping the link list
// current link node pointer is stored in $rax
// the link list's header is stored in %rdx

(gdb) info registers rax
rax            0x6462316231347830       7233398249312516144 // 
0x6462316231347830 !!!??? it is string "0x41b1bd"

(gdb) info registers rdx    // let's track the link list
rdx            0x26a5858        40523864
(gdb) x/x 0x26a5858 
0x26a5858:      0x00007fffd966f5e8
(gdb) x/x 0x00007fffd966f5e8
0x7fffd966f5e8: 0x0000000002781f78
(gdb) x/x 0x0000000002781f78
0x2781f78:      0x6462316231347830
(gdb) x/s 0x0000000002781f78
0x2781f78:       "0x41b1bdc4"  // "0x41b1bdc4" again

It crashes when copying from a broken link list, one node's _next ptr is set to 
a string address.
I am not familiar with breakpad's source code, so I have not found how to fix 
the problem.

There is something interesting, file ./minidump_stackwalk.32 (built on Linux 
32bit) won't crash on Linux x64.

$ ./minidump_stackwalk.32 crash.dump symbols > /dev/null 2>errorlog
works fine.

What is the expected output? What do you see instead?

Crashed.

What version of the product are you using? On what operating system?
$ svn info
Path: .
URL: http://google-breakpad.googlecode.com/svn/trunk
Repository Root: http://google-breakpad.googlecode.com/svn
Repository UUID: 4c0a9323-5329-0410-9bdc-e9ce6186880e
Revision: 1201
Node Kind: directory
Schedule: normal
Last Changed Author: yuwu@google.com
Last Changed Rev: 1201
Last Changed Date: 2013-07-16 02:33:10 +0800 (Tue, 16 Jul 2013)

$ uname -a 
Linux 10.1.1.101 3.5.7 #2 SMP Tue Jan 29 17:30:27 CST 2013 x86_64 GNU/Linux

Please provide any additional information below.

Original issue reported on code.google.com by at20042...@gmail.com on 24 Jul 2013 at 10:35

• Merged into: #474

Attachments:

• crash.tar.gz

[GoogleCodeExporter]

Thanks for your attentions!

Original comment by at20042...@gmail.com on 24 Jul 2013 at 10:37

[GoogleCodeExporter]

What version of gcc did you build minidump_stackwalk with? I assume it built 
with -O2 ?

Original comment by thestig@chromium.org on 24 Jul 2013 at 9:02

[GoogleCodeExporter]

You are right!
I have these in generated Makefile:
CCASFLAGS = -g -O2
CFLAGS = -g -O2
CXXFLAGS = -g -O2

After manually removing -O2, it works.
Thank you!

And more for this, in *README* file, this is one line:
CXXFLAGS=-m32 CFLAGS=-m32 CPPFLAGS=-m32 ./configure

This failed on Linux x64. And then I used only ./configure. It brings in -O2.
Now I use
CXXFLAGS=-m64 CFLAGS=-m64 CPPFLAGS=-m64 ./configure
or simply
CXXFLAGS= CFLAGS= CPPFLAGS= ./configure

Both work.

Thank you again!

Original comment by at20042...@gmail.com on 25 Jul 2013 at 2:49

[GoogleCodeExporter]

You never mentioned what version of gcc you are using. I suspect this is 
another case of bug 474.

Original comment by thestig@chromium.org on 25 Jul 2013 at 3:10

[GoogleCodeExporter]

sorry!

Here it is.

$ gcc --version
gcc (Debian 4.3.2-1.1) 4.3.2
Copyright (C) 2008 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Original comment by at20042...@gmail.com on 25 Jul 2013 at 3:58

[GoogleCodeExporter]

Original comment by thestig@chromium.org on 25 Jul 2013 at 8:17

• Changed state: Duplicate