arvindm95 / unladen-swallow

Automatically exported from code.google.com/p/unladen-swallow
Other
0 stars 0 forks source link

New segfault on x86-64 #85

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 8 years ago
The buildslaves are showing a segfault on x86-64 in both debug and release 
mode. This is a different problem than that fixed for issue 83 (which exposed 
this new segfault), though this has similar symptoms: x86-64 only, no pattern 
to test failures (ie, it's not test_foo failing every time), difficulty 
reproducing 
by hand.

Original issue reported on code.google.com by collinw on 9 Oct 2009 at 6:05

GoogleCodeExporter commented 8 years ago
I've tried disabling the "direct calls to C functions" optimization (which 
caused the last 
segfault); the segfault persists.

Original comment by collinw on 9 Oct 2009 at 9:24

GoogleCodeExporter commented 8 years ago
I've finally gotten a setup where I can reproduce the segfault reliably: Ubuntu 
Hardy, x86-64, 64-bit binary, gcc 4.2.4, logged in as root, run via 
"make buildbottest" (running regrtest.py with the same options doesn't work). 
This is a release build of both Unladen and LLVM.

Running Python under gdb causes the problem to go away, so I'm setting ulimit 
-c unlimited and PYTHONLLVMFLAGS="-jit-emit-debug" and 
debugging the core file.

(gdb) bt
#0  0x00002aaaa5c810b0 in ?? ()
#1  0x00002aaab33482d2 in _23_u_23___init__45 ()
#2  0x0000000000569530 in PyEval_EvalFrame (f=0x3dcdbf0) at Python/eval.cc:933
#3  0x000000000056fc1d in fast_function (stack_pointer=0x42bafa8, na=2, 
nk=<value optimized out>) at Python/eval.cc:4278
#4  _PyEval_CallFunction (stack_pointer=0x42bafa8, na=2, nk=<value optimized 
out>) at Python/eval.cc:4166
#5  0x00000000005699b8 in PyEval_EvalFrame (f=0x42badd0) at Python/eval.cc:2623
... lots more frames.

(gdb) up
#1  0x00002aaab33482d2 in _23_u_23___init__45 ()
(gdb) disas
Dump of assembler code for function _23_u_23___init__45:
0x00002aaab3348010 <_23_u_23___init__45+0>: push   %rbp
0x00002aaab3348011 <_23_u_23___init__45+1>: push   %r15
0x00002aaab3348013 <_23_u_23___init__45+3>: push   %r14
0x00002aaab3348015 <_23_u_23___init__45+5>: push   %r13
0x00002aaab3348017 <_23_u_23___init__45+7>: push   %r12
0x00002aaab3348019 <_23_u_23___init__45+9>: push   %rbx
0x00002aaab334801a <_23_u_23___init__45+10>:    sub    $0x28,%rsp
0x00002aaab334801e <_23_u_23___init__45+14>:    mov    0x40(%rdi),%rax
0x00002aaab3348022 <_23_u_23___init__45+18>:    mov    %rax,0x18(%rsp)
0x00002aaab3348027 <_23_u_23___init__45+23>:    mov    $0x1089620,%rax
0x00002aaab3348031 <_23_u_23___init__45+33>:    mov    (%rax),%rax
0x00002aaab3348034 <_23_u_23___init__45+36>:    mov    %rax,0x10(%rsp)
0x00002aaab3348039 <_23_u_23___init__45+41>:    movq   $0x0,0x48(%rdi)
0x00002aaab3348041 <_23_u_23___init__45+49>:    cmpl   $0x0,0x20(%rax)
0x00002aaab3348045 <_23_u_23___init__45+53>:    mov    %rdi,0x20(%rsp)
0x00002aaab334804a <_23_u_23___init__45+58>:    je     0x2aaab3348281 
<_23_u_23___init__45+625>
0x00002aaab3348050 <_23_u_23___init__45+64>:    mov    0x20(%rsp),%rbx
0x00002aaab3348055 <_23_u_23___init__45+69>:    movb   $0x1,0x86(%rbx)
0x00002aaab334805c <_23_u_23___init__45+76>:    mov    0x20(%rsp),%rax
0x00002aaab3348061 <_23_u_23___init__45+81>:    movl   $0x0,0x7c(%rax)
0x00002aaab3348068 <_23_u_23___init__45+88>:    mov    0x18(%rsp),%rcx
0x00002aaab334806d <_23_u_23___init__45+93>:    mov    %rcx,0x48(%rax)
0x00002aaab3348071 <_23_u_23___init__45+97>:    movb   $0x0,0x85(%rax)
0x00002aaab3348078 <_23_u_23___init__45+104>:   mov    %rax,%rdi
0x00002aaab334807b <_23_u_23___init__45+107>:   callq  0x2aaaa5c81010
0x00002aaab3348080 <_23_u_23___init__45+112>:   jmpq   0x2aaab3348175 
<_23_u_23___init__45+357>
0x00002aaab3348085 <_23_u_23___init__45+117>:   mov    0x20(%rsp),%rdi
0x00002aaab334808a <_23_u_23___init__45+122>:   callq  0x2aaaa5c81020
0x00002aaab334808f <_23_u_23___init__45+127>:   mov    0x10(%rsp),%rax
0x00002aaab3348094 <_23_u_23___init__45+132>:   cmpq   $0x0,0x30(%rax)
0x00002aaab3348099 <_23_u_23___init__45+137>:   jne    0x2aaab3349310 
<_23_u_23___init__45+4864>
0x00002aaab334809f <_23_u_23___init__45+143>:   xor    %r14d,%r14d
0x00002aaab33480a2 <_23_u_23___init__45+146>:   mov    $0x2,%r15b
0x00002aaab33480a5 <_23_u_23___init__45+149>:   mov    %r15b,%r12b
0x00002aaab33480a8 <_23_u_23___init__45+152>:   cmp    0x18(%rsp),%rbx
0x00002aaab33480ad <_23_u_23___init__45+157>:   jbe    0x2aaab33480e2 
<_23_u_23___init__45+210>
0x00002aaab33480b3 <_23_u_23___init__45+163>:   mov    -0x8(%rbx),%rdi
0x00002aaab33480b7 <_23_u_23___init__45+167>:   test   %rdi,%rdi
0x00002aaab33480ba <_23_u_23___init__45+170>:   je     0x2aaab33480d9 
<_23_u_23___init__45+201>
0x00002aaab33480c0 <_23_u_23___init__45+176>:   mov    (%rdi),%rax
0x00002aaab33480c3 <_23_u_23___init__45+179>:   dec    %rax
0x00002aaab33480c6 <_23_u_23___init__45+182>:   mov    %rax,(%rdi)
0x00002aaab33480c9 <_23_u_23___init__45+185>:   test   %rax,%rax
0x00002aaab33480cc <_23_u_23___init__45+188>:   jne    0x2aaab33480d9 
<_23_u_23___init__45+201>
0x00002aaab33480d2 <_23_u_23___init__45+194>:   mov    0x8(%rdi),%rax
0x00002aaab33480d6 <_23_u_23___init__45+198>:   callq  *0x30(%rax)
0x00002aaab33480d9 <_23_u_23___init__45+201>:   add    $0xfffffffffffffff8,%rbx
0x00002aaab33480dd <_23_u_23___init__45+205>:   jmpq   0x2aaab33480a8 
<_23_u_23___init__45+152>
0x00002aaab33480e2 <_23_u_23___init__45+210>:   xor    %ebx,%ebx
0x00002aaab33480e4 <_23_u_23___init__45+212>:   cmp    $0x8,%r12b
0x00002aaab33480e8 <_23_u_23___init__45+216>:   cmove  %r14,%rbx
0x00002aaab33480ec <_23_u_23___init__45+220>:   mov    0x10(%rsp),%r14
0x00002aaab33480f1 <_23_u_23___init__45+225>:   cmpl   $0x0,0x20(%r14)
0x00002aaab33480f6 <_23_u_23___init__45+230>:   je     0x2aaab3348154 
<_23_u_23___init__45+324>
0x00002aaab33480fc <_23_u_23___init__45+236>:   cmp    $0x8,%r15b
0x00002aaab3348100 <_23_u_23___init__45+240>:   sete   %r12b
0x00002aaab3348104 <_23_u_23___init__45+244>:   movzbl %r12b,%ecx
0x00002aaab3348108 <_23_u_23___init__45+248>:   cmp    $0x2,%r15b
0x00002aaab334810c <_23_u_23___init__45+252>:   sete   %r15b
0x00002aaab3348110 <_23_u_23___init__45+256>:   movzbl %r15b,%r8d
0x00002aaab3348114 <_23_u_23___init__45+260>:   mov    %r14,%rdi
0x00002aaab3348117 <_23_u_23___init__45+263>:   mov    0x20(%rsp),%rsi
0x00002aaab334811c <_23_u_23___init__45+268>:   mov    %rbx,%rdx
0x00002aaab334811f <_23_u_23___init__45+271>:   callq  0x2aaaa5c81030
0x00002aaab3348124 <_23_u_23___init__45+276>:   test   %eax,%eax
0x00002aaab3348126 <_23_u_23___init__45+278>:   je     0x2aaab3348154 
<_23_u_23___init__45+324>
0x00002aaab334812c <_23_u_23___init__45+284>:   test   %rbx,%rbx
0x00002aaab334812f <_23_u_23___init__45+287>:   je     0x2aaab3348152 
<_23_u_23___init__45+322>
0x00002aaab3348135 <_23_u_23___init__45+293>:   mov    (%rbx),%r14
0x00002aaab3348138 <_23_u_23___init__45+296>:   dec    %r14
0x00002aaab334813b <_23_u_23___init__45+299>:   mov    %r14,(%rbx)
0x00002aaab334813e <_23_u_23___init__45+302>:   test   %r14,%r14
0x00002aaab3348141 <_23_u_23___init__45+305>:   jne    0x2aaab3348152 
<_23_u_23___init__45+322>
0x00002aaab3348147 <_23_u_23___init__45+311>:   mov    0x8(%rbx),%r14
0x00002aaab334814b <_23_u_23___init__45+315>:   mov    %rbx,%rdi
0x00002aaab334814e <_23_u_23___init__45+318>:   callq  *0x30(%r14)
0x00002aaab3348152 <_23_u_23___init__45+322>:   xor    %ebx,%ebx
0x00002aaab3348154 <_23_u_23___init__45+324>:   mov    0x10(%rsp),%r14
0x00002aaab3348159 <_23_u_23___init__45+329>:   mov    0x10(%r14),%r14
0x00002aaab334815d <_23_u_23___init__45+333>:   cmpq   $0x0,0x58(%r14)
0x00002aaab3348162 <_23_u_23___init__45+338>:   je     0x2aaab3348172 
<_23_u_23___init__45+354>
0x00002aaab3348168 <_23_u_23___init__45+344>:   mov    0x10(%rsp),%rdi
0x00002aaab334816d <_23_u_23___init__45+349>:   callq  0x2aaaa5c81040
0x00002aaab3348172 <_23_u_23___init__45+354>:   mov    %rbx,%rax
0x00002aaab3348175 <_23_u_23___init__45+357>:   add    $0x28,%rsp
0x00002aaab3348179 <_23_u_23___init__45+361>:   pop    %rbx
0x00002aaab334817a <_23_u_23___init__45+362>:   pop    %r12
0x00002aaab334817c <_23_u_23___init__45+364>:   pop    %r13
0x00002aaab334817e <_23_u_23___init__45+366>:   pop    %r14
0x00002aaab3348180 <_23_u_23___init__45+368>:   pop    %r15
0x00002aaab3348182 <_23_u_23___init__45+370>:   pop    %rbp
0x00002aaab3348183 <_23_u_23___init__45+371>:   retq   
0x00002aaab3348184 <_23_u_23___init__45+372>:   mov    $0x106158c,%r13
0x00002aaab334818e <_23_u_23___init__45+382>:   mov    0x0(%r13),%eax
0x00002aaab3348192 <_23_u_23___init__45+386>:   dec    %eax
0x00002aaab3348194 <_23_u_23___init__45+388>:   mov    %eax,0x0(%r13)
0x00002aaab3348198 <_23_u_23___init__45+392>:   test   %eax,%eax
0x00002aaab334819a <_23_u_23___init__45+394>:   jns    0x2aaab33481b2 
<_23_u_23___init__45+418>
0x00002aaab33481a0 <_23_u_23___init__45+400>:   mov    0x10(%rsp),%rdi
0x00002aaab33481a5 <_23_u_23___init__45+405>:   callq  0x2aaaa5c810f0
0x00002aaab33481aa <_23_u_23___init__45+410>:   test   %eax,%eax
0x00002aaab33481ac <_23_u_23___init__45+412>:   js     0x2aaab3348085 
<_23_u_23___init__45+117>
0x00002aaab33481b2 <_23_u_23___init__45+418>:   mov    0x20(%rsp),%r13
0x00002aaab33481b7 <_23_u_23___init__45+423>:   movl   $0xce,0x80(%r13)
0x00002aaab33481c2 <_23_u_23___init__45+434>:   mov    $0x1089220,%r13
0x00002aaab33481cc <_23_u_23___init__45+444>:   cmpl   $0x0,0x0(%r13)
0x00002aaab33481d1 <_23_u_23___init__45+449>:   jne    0x2aaab33492f3 
<_23_u_23___init__45+4835>
0x00002aaab33481d7 <_23_u_23___init__45+455>:   mov    0x18(%rsp),%r13
0x00002aaab33481dc <_23_u_23___init__45+460>:   mov    0x0(%r13),%r13
0x00002aaab33481e0 <_23_u_23___init__45+464>:   mov    %r15,%rbp
0x00002aaab33481e3 <_23_u_23___init__45+467>:   mov    0x8(%r13),%r15
0x00002aaab33481e7 <_23_u_23___init__45+471>:   mov    %r13,%rdi
0x00002aaab33481ea <_23_u_23___init__45+474>:   callq  *0xe0(%r15)
0x00002aaab33481f1 <_23_u_23___init__45+481>:   test   %rax,%rax
0x00002aaab33481f4 <_23_u_23___init__45+484>:   je     0x2aaab3348cbb 
<_23_u_23___init__45+3243>
0x00002aaab33481fa <_23_u_23___init__45+490>:   mov    %rax,%r15
0x00002aaab33481fd <_23_u_23___init__45+493>:   mov    0x18(%rsp),%rax
0x00002aaab3348202 <_23_u_23___init__45+498>:   mov    %r13,(%rax)
0x00002aaab3348205 <_23_u_23___init__45+501>:   mov    %r15,0x8(%rax)
0x00002aaab3348209 <_23_u_23___init__45+505>:   mov    0x20(%rsp),%r13
0x00002aaab334820e <_23_u_23___init__45+510>:   mov    %r15,0x188(%r13)
0x00002aaab3348215 <_23_u_23___init__45+517>:   test   %rbp,%rbp
0x00002aaab3348218 <_23_u_23___init__45+520>:   je     0x2aaab334823d 
<_23_u_23___init__45+557>
0x00002aaab334821e <_23_u_23___init__45+526>:   mov    0x0(%rbp),%r13
0x00002aaab3348222 <_23_u_23___init__45+530>:   dec    %r13
0x00002aaab3348225 <_23_u_23___init__45+533>:   mov    %r13,0x0(%rbp)
0x00002aaab3348229 <_23_u_23___init__45+537>:   test   %r13,%r13
0x00002aaab334822c <_23_u_23___init__45+540>:   jne    0x2aaab334823d 
<_23_u_23___init__45+557>
0x00002aaab3348232 <_23_u_23___init__45+546>:   mov    0x8(%rbp),%r13
0x00002aaab3348236 <_23_u_23___init__45+550>:   mov    %rbp,%rdi
0x00002aaab3348239 <_23_u_23___init__45+553>:   callq  *0x30(%r13)
0x00002aaab334823d <_23_u_23___init__45+557>:   mov    0x20(%rsp),%r13
0x00002aaab3348242 <_23_u_23___init__45+562>:   movl   $0xcf,0x80(%r13)
0x00002aaab334824d <_23_u_23___init__45+573>:   mov    $0x1089220,%r13
0x00002aaab3348257 <_23_u_23___init__45+583>:   cmpl   $0x0,0x0(%r13)
0x00002aaab334825c <_23_u_23___init__45+588>:   je     0x2aaab3348d45 
<_23_u_23___init__45+3381>
0x00002aaab3348262 <_23_u_23___init__45+594>:   mov    0x20(%rsp),%r14
0x00002aaab3348267 <_23_u_23___init__45+599>:   movl   $0x84,0x78(%r14)
0x00002aaab334826f <_23_u_23___init__45+607>:   movb   $0x2,0x86(%r14)
0x00002aaab3348277 <_23_u_23___init__45+615>:   mov    %rbx,0x18(%rsp)
0x00002aaab334827c <_23_u_23___init__45+620>:   jmpq   0x2aaab334805c 
<_23_u_23___init__45+76>
0x00002aaab3348281 <_23_u_23___init__45+625>:   mov    0x178(%rdi),%r14
0x00002aaab3348288 <_23_u_23___init__45+632>:   mov    %rdi,%rbx
0x00002aaab334828b <_23_u_23___init__45+635>:   mov    0x20(%rbx),%rax
0x00002aaab334828f <_23_u_23___init__45+639>:   mov    %rax,0x8(%rsp)
0x00002aaab3348294 <_23_u_23___init__45+644>:   movl   $0xc2,0x80(%rbx)
0x00002aaab334829e <_23_u_23___init__45+654>:   mov    $0x1089220,%rbx
0x00002aaab33482a8 <_23_u_23___init__45+664>:   cmpl   $0x0,(%rbx)
0x00002aaab33482ab <_23_u_23___init__45+667>:   jne    0x2aaab3348305 
<_23_u_23___init__45+757>
0x00002aaab33482b1 <_23_u_23___init__45+673>:   incq   (%r14)
0x00002aaab33482b4 <_23_u_23___init__45+676>:   mov    0x18(%rsp),%rbx
0x00002aaab33482b9 <_23_u_23___init__45+681>:   mov    %r14,(%rbx)
0x00002aaab33482bc <_23_u_23___init__45+684>:   mov    $0x2dd8b48,%rbx
0x00002aaab33482c6 <_23_u_23___init__45+694>:   mov    0x18(%rbx),%rsi
0x00002aaab33482ca <_23_u_23___init__45+698>:   mov    %r14,%rdi
0x00002aaab33482cd <_23_u_23___init__45+701>:   callq  0x2aaaa5c810b0
0x00002aaab33482d2 <_23_u_23___init__45+706>:   mov    (%r14),%rbx
0x00002aaab33482d5 <_23_u_23___init__45+709>:   dec    %rbx
... asm continues

(gdb) x 0x2aaaa5c810b0
0x2aaaa5c810b0: Cannot access memory at address 0x2aaaa5c810b0

That callq  0x2aaaa5c810b0 instruction is the problem: it's calling to invalid 
memory. Looking at the other callq instructions in this file, gdb can't 
access any of them.

For comparison, gdb *can* access the callq targets in other functions:

(gdb) f 32
#32 0x00002b8ea5cca2f8 in _23_u_23_dispatch ()
(gdb) disas
Dump of assembler code for function _23_u_23_dispatch:
0x00002b8ea5cc99f0 <_23_u_23_dispatch+0>:   push   %rbp
0x00002b8ea5cc99f1 <_23_u_23_dispatch+1>:   push   %r15
0x00002b8ea5cc99f3 <_23_u_23_dispatch+3>:   push   %r14
0x00002b8ea5cc99f5 <_23_u_23_dispatch+5>:   push   %r13
0x00002b8ea5cc99f7 <_23_u_23_dispatch+7>:   push   %r12
0x00002b8ea5cc99f9 <_23_u_23_dispatch+9>:   push   %rbx
0x00002b8ea5cc99fa <_23_u_23_dispatch+10>:  sub    $0x48,%rsp
0x00002b8ea5cc99fe <_23_u_23_dispatch+14>:  mov    0x40(%rdi),%rbx
0x00002b8ea5cc9a02 <_23_u_23_dispatch+18>:  mov    $0x1089620,%rax
0x00002b8ea5cc9a0c <_23_u_23_dispatch+28>:  mov    (%rax),%rax
0x00002b8ea5cc9a0f <_23_u_23_dispatch+31>:  mov    %rax,0x38(%rsp)
0x00002b8ea5cc9a14 <_23_u_23_dispatch+36>:  movq   $0x0,0x48(%rdi)
0x00002b8ea5cc9a1c <_23_u_23_dispatch+44>:  cmpl   $0x0,0x20(%rax)
0x00002b8ea5cc9a20 <_23_u_23_dispatch+48>:  mov    %rdi,0x40(%rsp)
0x00002b8ea5cc9a25 <_23_u_23_dispatch+53>:  je     0x2b8ea5cc9b5e 
<_23_u_23_dispatch+366>
0x00002b8ea5cc9a2b <_23_u_23_dispatch+59>:  mov    0x40(%rsp),%r14
0x00002b8ea5cc9a30 <_23_u_23_dispatch+64>:  movb   $0x1,0x86(%r14)
0x00002b8ea5cc9a38 <_23_u_23_dispatch+72>:  mov    0x40(%rsp),%rax
0x00002b8ea5cc9a3d <_23_u_23_dispatch+77>:  movl   $0x0,0x7c(%rax)
0x00002b8ea5cc9a44 <_23_u_23_dispatch+84>:  mov    %rbx,0x48(%rax)
0x00002b8ea5cc9a48 <_23_u_23_dispatch+88>:  movb   $0x0,0x85(%rax)
0x00002b8ea5cc9a4f <_23_u_23_dispatch+95>:  mov    %rax,%rdi
0x00002b8ea5cc9a52 <_23_u_23_dispatch+98>:  callq  0x2b8ea5c81010
0x00002b8ea5cc9a57 <_23_u_23_dispatch+103>: jmpq   0x2b8ea5cc9b4f 
<_23_u_23_dispatch+351>
0x00002b8ea5cc9a5c <_23_u_23_dispatch+108>: mov    0x10(%rsp),%r14
0x00002b8ea5cc9a61 <_23_u_23_dispatch+113>: mov    0x40(%rsp),%rdi
0x00002b8ea5cc9a66 <_23_u_23_dispatch+118>: callq  0x2b8ea5c81020
0x00002b8ea5cc9a6b <_23_u_23_dispatch+123>: mov    0x38(%rsp),%rax
0x00002b8ea5cc9a70 <_23_u_23_dispatch+128>: cmpq   $0x0,0x30(%rax)
0x00002b8ea5cc9a75 <_23_u_23_dispatch+133>: jne    0x2b8ea5cca34c 
<_23_u_23_dispatch+2396>
0x00002b8ea5cc9a7b <_23_u_23_dispatch+139>: xor    %r15d,%r15d
0x00002b8ea5cc9a7e <_23_u_23_dispatch+142>: mov    $0x2,%r12b
0x00002b8ea5cc9a81 <_23_u_23_dispatch+145>: mov    %r12b,%r13b
0x00002b8ea5cc9a84 <_23_u_23_dispatch+148>: cmp    %rbx,%r14
0x00002b8ea5cc9a87 <_23_u_23_dispatch+151>: jbe    0x2b8ea5cc9abc 
<_23_u_23_dispatch+204>
0x00002b8ea5cc9a8d <_23_u_23_dispatch+157>: mov    -0x8(%r14),%rdi
0x00002b8ea5cc9a91 <_23_u_23_dispatch+161>: test   %rdi,%rdi
0x00002b8ea5cc9a94 <_23_u_23_dispatch+164>: je     0x2b8ea5cc9ab3 
<_23_u_23_dispatch+195>
0x00002b8ea5cc9a9a <_23_u_23_dispatch+170>: mov    (%rdi),%rax
0x00002b8ea5cc9a9d <_23_u_23_dispatch+173>: dec    %rax
0x00002b8ea5cc9aa0 <_23_u_23_dispatch+176>: mov    %rax,(%rdi)
0x00002b8ea5cc9aa3 <_23_u_23_dispatch+179>: test   %rax,%rax
0x00002b8ea5cc9aa6 <_23_u_23_dispatch+182>: jne    0x2b8ea5cc9ab3 
<_23_u_23_dispatch+195>
0x00002b8ea5cc9aac <_23_u_23_dispatch+188>: mov    0x8(%rdi),%rax
0x00002b8ea5cc9ab0 <_23_u_23_dispatch+192>: callq  *0x30(%rax)
---Type <return> to continue, or q <return> to quit---q
Quit
(gdb) x 0x2b8ea5c81010
0x2b8ea5c81010: 0x8f50ba49
(gdb) disas 0x2b8ea5c81010 0x2b8ea5c8101d
Dump of assembler code from 0x2b8ea5c81010 to 0x2b8ea5c8101d:
0x00002b8ea5c81010: mov    $0x568f50,%r10
0x00002b8ea5c8101a: jmpq   *%r10
End of assembler dump.
(gdb)

So that's a stub, which almost certainly means the bad callq is calling to a 
stub.

(gdb) p f->f_code->co_hotness
$3 = 100004

So this function has *just* become hot and gone through compilation. The 
function that's failing here is 
Lib/compiler/pycodegen.py:CodeGenerator.__init__(), but I've seen it fail in at 
least two other functions in the compiler package while running 
test_compiler, so I don't think it's anything particular to this function.

It's interesting that the bad callq is jumping far, far further to get to its 
stub than the good callq's I was looking at:

>>> 0x00002b8ea5d15008 - 0x2b8ea5c81010  # Good
606200L
>>> 0x00002aaab33482cd - 0x2aaaa5c810b0  # Busted
225210909L
>>>

Uninformed speculation: does LLVM actually need to emit chained stubs in this 
case?

Note to self: the core file is at ~/unladen-segfault.core.

Original comment by collinw on 13 Oct 2009 at 3:38

GoogleCodeExporter commented 8 years ago
OOOOH, idea:

The stubs and the code are located more than 32 bits of address space away.  You
can't encode immediate values larger than 2^32-1 in x86 asm, and it's getting
truncated.  The stubs should be laid out more or less consecutively.  Compare 
the
addresses:
0x2b8e a5c81010  # Good
0x2aaa a5c810b0  # Busted

They're the same in the low 32 bits, and should be more or less consecutive.  
Then
again, call addresses are encoded as IP-relative, so maybe that's wrong.

Step one, to check if this is correct, is to add an assertion to check that the
offset isn't going to get truncated.  Alternatively, the memory manager could 
assert
that the slab addresses it gets back are nearby.

The solution, on the LLVM JIT side, would be to fix the memory allocator again 
so
that it can get code and stubs all mapped into one 4 GB region of address 
space. 
Right now, it tries to lay things out consecutively, but if mmap returns a 
different
address, it just uses that one.

Original comment by reid.kle...@gmail.com on 13 Oct 2009 at 3:58

GoogleCodeExporter commented 8 years ago
Tweaking one of the DEBUG lines in JITMemoryManager.cpp to always be on 
produces this (this is a different test run than that above):

[...tests...]
test_importhooks
test_enumerate
test_getopt
test_codecencodings_cn
JIT: Allocating another slab of memory for function.make: *** [buildbottest] 
Segmentation fault

Adding assertions that the offset fits within 32 bits gives me this:

[...tests...]
test_importhooks
test_enumerate
test_getopt
test_codecencodings_cn
JIT: Allocating another slab of memory for function.
python: 
/usr/local/google/collinwinter/us/trunk/Util/llvm/lib/ExecutionEngine/JIT/JITEmi
tter.cpp:652: 
void*<unnamed>::JITEmitter::getPointerToGlobal(llvm::GlobalValue*, void*, 
bool): Assertion `(Offset & 0xFFFFFFFF) == Offset && "Offset 
too big for 32 bits"' failed.
Stack dump:
0.  Running pass 'X86 Machine Code Emitter' on function '@"#u#readline241"'
make: *** [buildbottest] Aborted (core dumped)

So Reid's theory is correct. I'm working on a reduced test case that I can send 
upstream.

Now that I'm confident I know what's going on, I'll file a bug with LLVM.

Original comment by collinw on 14 Oct 2009 at 11:51

GoogleCodeExporter commented 8 years ago
Tracking upstream at http://llvm.org/PR5201.

Original comment by collinw on 15 Oct 2009 at 6:49

GoogleCodeExporter commented 8 years ago
Test case sent upstream. Changing mmap to use MAP_32BIT fixes the test case; 
the 
buildslaves are testing whether this fixes things overall.

Original comment by collinw on 19 Oct 2009 at 10:31

GoogleCodeExporter commented 8 years ago
Jeffrey, does anything still need to be done for this? The buildbots say it's 
fixed, but I 
don't know if you have any further upstream LLVM patches that need to be pulled 
in to 
our tree.

Original comment by collinw on 9 Dec 2009 at 6:04

GoogleCodeExporter commented 8 years ago
I believe it's fixed.

Original comment by jyass...@gmail.com on 9 Dec 2009 at 7:24