v6.12.x segfault on linux

simevo commented 6 years ago

Version: v6.12.0 and v6.12.2
Platform: Linux workhorse5 4.13.0-1-amd64 #1 SMP Debian 4.13.13-1 (2017-11-16) x86_64
Subsystem: N/A

We encountered a segfault of nodejs 6.12.0, see debian bug #878674

To make it easy for others to reproduce it, I did this.

On a clean debian sid install, I install webpack and some babel stuff:

sudo apt install webpack node-babel-loader node-babel-cli node-babel-plugin-syntax-dynamic-import node-babel-plugin-add-module-exports node-babel-preset-es2015

Next I clone the WIP packaging repo for vue.js as it stands today:

git clone https://anonscm.debian.org/cgit/pkg-javascript/vue.js.git
cd vue.js
git checkout 886a44ec235b6c3236a124b37560f7a9765b6816

If I then run the command (using nodejs 6.12.0 from the debian archive):

webpack --config debian/webpack.config.js

I get:

Segmentation fault

It also segfaults when using node 6.12.2 executable downloaded from nodejs.org:

cd ..
wget https://nodejs.org/dist/latest-v6.x/node-v6.12.2-linux-x64.tar.xz
xz -d node-v6.12.2-linux-x64.tar.xz
tar xf node-v6.12.2-linux-x64.tar
cd vue.js
NODE_PATH=/usr/lib/nodejs/ ../node-v6.12.2-linux-x64/bin/node /usr/lib/nodejs/webpack/bin/webpack.js --config debian/webpack.config.js 
Segmentation fault

simevo commented 6 years ago

In addition to the steps above, to reproduce the segfault even if you are not on debian sid I believe the content of the /usr/lib/nodejs directory should suffice.

Here is a copy of the content of that directory on my test VM: http://212.237.56.209/node_modules.tar.xz (1.5 MB)

Let me know if there is some other steps I should try. And THANKS !

bnoordhuis commented 6 years ago

I see in the linked issue you've been debugging with gdb. Can you post the output of info sharedlibrary at the time of the crash with the binary from https://nodejs.org/?

simevo commented 6 years ago

NODE_PATH=/usr/lib/nodejs/ gdb ../node-v6.12.2-linux-x64/bin/node
r /usr/lib/nodejs/webpack/bin/webpack.js --config debian/webpack.config.js
info sharedlibrary

yields:

From                To                  Syms Read   Shared Object Library
0x00007ffff7ddaaa0  0x00007ffff7df6070  Yes         /lib64/ld-linux-x86-64.so.2
0x00007ffff7bd6e90  0x00007ffff7bd7b4e  Yes         /lib/x86_64-linux-gnu/libdl.so.2
0x00007ffff79d0190  0x00007ffff79d32af  Yes         /lib/x86_64-linux-gnu/librt.so.1
0x00007ffff76db090  0x00007ffff7783b69  Yes (*)     /usr/lib/x86_64-linux-gnu/libstdc++.so.6
0x00007ffff7341f20  0x00007ffff73bbd2a  Yes         /lib/x86_64-linux-gnu/libm.so.6
0x00007ffff7127ac0  0x00007ffff7137de5  Yes (*)     /lib/x86_64-linux-gnu/libgcc_s.so.1
0x00007ffff6f0cb10  0x00007ffff6f1abc1  Yes         /lib/x86_64-linux-gnu/libpthread.so.0
0x00007ffff6b83b90  0x00007ffff6cb43b3  Yes         /lib/x86_64-linux-gnu/libc.so.6
(*): Shared library is missing debugging information.

hope this helps !

bnoordhuis commented 6 years ago

Thanks, nothing out of the ordinary. The backtraces from the debian bug report look like random memory corruption. Can you try a debug build and see if it catches it closer to the source?

Here is how you build it:

download the v6.12.2 tarball
./configure && make -j8 -C out BUILDTYPE=Debug
out/Debug/node is the debug binary

If you hit a run-time assert (a CHECK), please post it. If it still segfaults, can you post the output of thread apply all backtrace full and the output of disassemble and info registers? Thanks.

simevo commented 6 years ago

built with:

git clone https://github.com/nodejs/node.git
cd node
git checkout v6.12.2
./configure && make -j8 -C out BUILDTYPE=Debug

reproduced the segfault with:

cd ../vue.js
NODE_PATH=/usr/lib/nodejs/ gdb ../node/out/Debug/node 
r /usr/lib/nodejs/webpack/bin/webpack.js --config debian/webpack.config.js

output of thread apply all backtrace full:

Thread 10 (Thread 0x7fffe6ffd700 (LWP 12870)):
#0  0x00007ffff6f14b26 in futex_wait_cancelable (private=<optimized out>, expected=0, 
    futex_word=0x5555580e1bec <cond+44>) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
        __ret = -512
        oldtype = 0
        err = <optimized out>
#1  __pthread_cond_wait_common (abstime=0x0, mutex=0x5555580e1c00 <mutex>, 
    cond=0x5555580e1bc0 <cond>) at pthread_cond_wait.c:502
        spin = 0
        buffer = {__routine = 0x7ffff6f14960 <__condvar_cleanup_waiting>, 
          __arg = 0x7fffe6ffce40, __canceltype = -419443072, __prev = 0x0}
        cbuffer = {wseq = 627, cond = 0x5555580e1bc0 <cond>, 
          mutex = 0x5555580e1c00 <mutex>, private = 0}
        err = <optimized out>
        g = 1
        flags = <optimized out>
        signals = 0
        result = 0
        seq = 313
#2  __pthread_cond_wait (cond=0x5555580e1bc0 <cond>, mutex=0x5555580e1c00 <mutex>)
    at pthread_cond_wait.c:655
No locals.
#3  0x0000555557271ab5 in uv_cond_wait (cond=0x5555580e1bc0 <cond>, 
    mutex=0x5555580e1c00 <mutex>) at ../deps/uv/src/unix/thread.c:641
No locals.
#4  0x000055555725c182 in worker (arg=0x0) at ../deps/uv/src/threadpool.c:63
        w = 0x5555582228c8
        q = 0x5555582228e0
#5  0x00007ffff6f0e519 in start_thread (arg=0x7fffe6ffd700) at pthread_create.c:456
        __res = <optimized out>
        pd = 0x7fffe6ffd700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737068914432, 3140752450534644803, 
                140737488336590, 140737488336591, 7, 140737068914432, -3140701874591753149, 
                -3140736997479801789}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 
              0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#6  0x00007ffff6c50a4f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
No locals.

Thread 9 (Thread 0x7fffe77fe700 (LWP 12869)):
#0  0x00007ffff6f14b26 in futex_wait_cancelable (private=<optimized out>, expected=0, 
    futex_word=0x5555580e1be8 <cond+40>) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
        __ret = -512
        oldtype = 0
        err = <optimized out>
#1  __pthread_cond_wait_common (abstime=0x0, mutex=0x5555580e1c00 <mutex>, 
    cond=0x5555580e1bc0 <cond>) at pthread_cond_wait.c:502
        spin = 0
        buffer = {__routine = 0x7ffff6f14960 <__condvar_cleanup_waiting>, 
          __arg = 0x7fffe77fde40, __canceltype = -411050368, __prev = 0x0}
        cbuffer = {wseq = 622, cond = 0x5555580e1bc0 <cond>, 
          mutex = 0x5555580e1c00 <mutex>, private = 0}
        err = <optimized out>
        g = 0
        flags = <optimized out>
        signals = 0
        result = 0
        seq = 311
#2  __pthread_cond_wait (cond=0x5555580e1bc0 <cond>, mutex=0x5555580e1c00 <mutex>)
    at pthread_cond_wait.c:655
No locals.
#3  0x0000555557271ab5 in uv_cond_wait (cond=0x5555580e1bc0 <cond>, 
    mutex=0x5555580e1c00 <mutex>) at ../deps/uv/src/unix/thread.c:641
No locals.
#4  0x000055555725c182 in worker (arg=0x0) at ../deps/uv/src/threadpool.c:63
        w = 0x555558133568
        q = 0x555558133580
#5  0x00007ffff6f0e519 in start_thread (arg=0x7fffe77fe700) at pthread_create.c:456
        __res = <optimized out>
        pd = 0x7fffe77fe700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737077307136, 3140752450534644803, 
                140737488336590, 140737488336591, 7, 140737077307136, -3140698575519998909, 
                -3140736997479801789}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 
              0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#6  0x00007ffff6c50a4f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
No locals.

Thread 8 (Thread 0x7fffe7fff700 (LWP 12868)):
#0  0x00007ffff6f14b26 in futex_wait_cancelable (private=<optimized out>, expected=0, 
    futex_word=0x5555580e1bec <cond+44>) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
        __ret = -512
        oldtype = 0
        err = <optimized out>
#1  __pthread_cond_wait_common (abstime=0x0, mutex=0x5555580e1c00 <mutex>, 
    cond=0x5555580e1bc0 <cond>) at pthread_cond_wait.c:502
        spin = 0
        buffer = {__routine = 0x7ffff6f14960 <__condvar_cleanup_waiting>, 
          __arg = 0x7fffe7ffee40, __canceltype = -402657664, __prev = 0x0}
        cbuffer = {wseq = 629, cond = 0x5555580e1bc0 <cond>, 
          mutex = 0x5555580e1c00 <mutex>, private = 0}
        err = <optimized out>
        g = 1
        flags = <optimized out>
        signals = 0
        result = 0
        seq = 314
#2  __pthread_cond_wait (cond=0x5555580e1bc0 <cond>, mutex=0x5555580e1c00 <mutex>)
    at pthread_cond_wait.c:655
No locals.
#3  0x0000555557271ab5 in uv_cond_wait (cond=0x5555580e1bc0 <cond>, 
    mutex=0x5555580e1c00 <mutex>) at ../deps/uv/src/unix/thread.c:641
No locals.
#4  0x000055555725c182 in worker (arg=0x0) at ../deps/uv/src/threadpool.c:63
        w = 0x555558222cf8
        q = 0x555558222d10
#5  0x00007ffff6f0e519 in start_thread (arg=0x7fffe7fff700) at pthread_create.c:456
        __res = <optimized out>
        pd = 0x7fffe7fff700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737085699840, 3140752450534644803, 
                140737488336590, 140737488336591, 7, 140737085699840, -3140699674494755773, 
                -3140736997479801789}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 
              0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#6  0x00007ffff6c50a4f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
No locals.

Thread 7 (Thread 0x7ffff4b5f700 (LWP 12867)):
#0  0x00007ffff6f14b26 in futex_wait_cancelable (private=<optimized out>, expected=0, 
    futex_word=0x5555580e1bec <cond+44>) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
        __ret = -512
        oldtype = 0
        err = <optimized out>
#1  __pthread_cond_wait_common (abstime=0x0, mutex=0x5555580e1c00 <mutex>, 
    cond=0x5555580e1bc0 <cond>) at pthread_cond_wait.c:502
        spin = 0
        buffer = {__routine = 0x7ffff6f14960 <__condvar_cleanup_waiting>, 
          __arg = 0x7ffff4b5ee40, __canceltype = -189403568, __prev = 0x0}
        cbuffer = {wseq = 625, cond = 0x5555580e1bc0 <cond>, 
          mutex = 0x5555580e1c00 <mutex>, private = 0}
        err = <optimized out>
        g = 1
        flags = <optimized out>
        signals = 0
        result = 0
        seq = 312
#2  __pthread_cond_wait (cond=0x5555580e1bc0 <cond>, mutex=0x5555580e1c00 <mutex>)
    at pthread_cond_wait.c:655
No locals.
#3  0x0000555557271ab5 in uv_cond_wait (cond=0x5555580e1bc0 <cond>, 
    mutex=0x5555580e1c00 <mutex>) at ../deps/uv/src/unix/thread.c:641
No locals.
#4  0x000055555725c182 in worker (arg=0x0) at ../deps/uv/src/threadpool.c:63
        w = 0x5555581c2af8
        q = 0x5555581c2b10
#5  0x00007ffff6f0e519 in start_thread (arg=0x7ffff4b5f700) at pthread_create.c:456
        __res = <optimized out>
        pd = 0x7ffff4b5f700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737298953984, 3140752450534644803, 
                140737488336590, 140737488336591, 7, 140737298953984, -3140740820281451453, 
                -3140736997479801789}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 
              0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#6  0x00007ffff6c50a4f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
No locals.

Thread 6 (Thread 0x7ffff5360700 (LWP 12866)):
#0  0x00007ffff6f17666 in futex_abstimed_wait_cancelable (private=0, abstime=0x0, 
    expected=0, futex_word=0x5555580f80f8)
    at ../sysdeps/unix/sysv/linux/futex-internal.h:205
        __ret = -512
        oldtype = 0
        err = <optimized out>
#1  do_futex_wait (sem=sem@entry=0x5555580f80f8, abstime=0x0) at sem_waitcommon.c:111
No locals.
#2  0x00007ffff6f17724 in __new_sem_wait_slow (sem=0x5555580f80f8, abstime=0x0)
    at sem_waitcommon.c:181
        _buffer = {__routine = 0x7ffff6f17620 <__sem_wait_cleanup>, __arg = 0x5555580f80f8, 
          __canceltype = -181010960, __prev = 0x0}
        err = <optimized out>
        d = 12884901888
#3  0x000055555727d166 in v8::base::Semaphore::Wait (this=0x5555580f80f8)
    at ../deps/v8/src/base/platform/semaphore.cc:103
        result = 32767
#4  0x000055555707b41c in v8::platform::TaskQueue::GetNext (this=0x5555580f80c8)
    at ../deps/v8/src/libplatform/task-queue.cc:44
No locals.
#5  0x000055555707b590 in v8::platform::WorkerThread::Run (this=0x5555580f8940)
    at ../deps/v8/src/libplatform/worker-thread.cc:25
        task = 0x5555581388a0
#6  0x000055555727f14a in v8::base::Thread::NotifyStartedAndRun (this=0x5555580f8940)
    at ../deps/v8/src/base/platform/platform.h:472
No locals.
#7  0x000055555727ff6c in v8::base::ThreadEntry (arg=0x5555580f8940)
    at ../deps/v8/src/base/platform/platform-posix.cc:600
        thread = 0x5555580f8940
#8  0x00007ffff6f0e519 in start_thread (arg=0x7ffff5360700) at pthread_create.c:456
        __res = <optimized out>
        pd = 0x7ffff5360700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737307346688, 3140752450534644803, 
                140737488347454, 140737488347455, 0, 140737307346688, -3140737521209697213, 
                -3140736997479801789}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 
              0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#9  0x00007ffff6c50a4f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
No locals.

Thread 5 (Thread 0x7ffff5b61700 (LWP 12865)):
#0  0x00007ffff6f17666 in futex_abstimed_wait_cancelable (private=0, abstime=0x0, 
    expected=0, futex_word=0x5555580f80f8)
    at ../sysdeps/unix/sysv/linux/futex-internal.h:205
        __ret = -512
        oldtype = 0
        err = <optimized out>
#1  do_futex_wait (sem=sem@entry=0x5555580f80f8, abstime=0x0) at sem_waitcommon.c:111
No locals.
#2  0x00007ffff6f17724 in __new_sem_wait_slow (sem=0x5555580f80f8, abstime=0x0)
    at sem_waitcommon.c:181
        _buffer = {__routine = 0x7ffff6f17620 <__sem_wait_cleanup>, __arg = 0x5555580f80f8, 
          __canceltype = -172618256, __prev = 0x0}
        err = <optimized out>
        d = 12884901888
#3  0x000055555727d166 in v8::base::Semaphore::Wait (this=0x5555580f80f8)
    at ../deps/v8/src/base/platform/semaphore.cc:103
        result = 32767
#4  0x000055555707b41c in v8::platform::TaskQueue::GetNext (this=0x5555580f80c8)
    at ../deps/v8/src/libplatform/task-queue.cc:44
No locals.
#5  0x000055555707b590 in v8::platform::WorkerThread::Run (this=0x5555580f8790)
    at ../deps/v8/src/libplatform/worker-thread.cc:25
        task = 0x5555581331c0
#6  0x000055555727f14a in v8::base::Thread::NotifyStartedAndRun (this=0x5555580f8790)
    at ../deps/v8/src/base/platform/platform.h:472
No locals.
#7  0x000055555727ff6c in v8::base::ThreadEntry (arg=0x5555580f8790)
    at ../deps/v8/src/base/platform/platform-posix.cc:600
        thread = 0x5555580f8790
#8  0x00007ffff6f0e519 in start_thread (arg=0x7ffff5b61700) at pthread_create.c:456
        __res = <optimized out>
        pd = 0x7ffff5b61700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737315739392, 3140752450534644803, 
                140737488347454, 140737488347455, 0, 140737315739392, -3140738633069355965, 
                -3140736997479801789}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 
              0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#9  0x00007ffff6c50a4f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
No locals.

Thread 4 (Thread 0x7ffff6362700 (LWP 12864)):
#0  0x00007ffff6f17666 in futex_abstimed_wait_cancelable (private=0, abstime=0x0, 
    expected=0, futex_word=0x5555580f80f8)
    at ../sysdeps/unix/sysv/linux/futex-internal.h:205
        __ret = -512
        oldtype = 0
        err = <optimized out>
#1  do_futex_wait (sem=sem@entry=0x5555580f80f8, abstime=0x0) at sem_waitcommon.c:111
No locals.
#2  0x00007ffff6f17724 in __new_sem_wait_slow (sem=0x5555580f80f8, abstime=0x0)
    at sem_waitcommon.c:181
        _buffer = {__routine = 0x7ffff6f17620 <__sem_wait_cleanup>, __arg = 0x5555580f80f8, 
          __canceltype = -164225552, __prev = 0x0}
        err = <optimized out>
        d = 12884901888
#3  0x000055555727d166 in v8::base::Semaphore::Wait (this=0x5555580f80f8)
    at ../deps/v8/src/base/platform/semaphore.cc:103
        result = 32767
#4  0x000055555707b41c in v8::platform::TaskQueue::GetNext (this=0x5555580f80c8)
    at ../deps/v8/src/libplatform/task-queue.cc:44
No locals.
#5  0x000055555707b590 in v8::platform::WorkerThread::Run (this=0x5555580f85e0)
    at ../deps/v8/src/libplatform/worker-thread.cc:25
        task = 0x55555814bd40
#6  0x000055555727f14a in v8::base::Thread::NotifyStartedAndRun (this=0x5555580f85e0)
    at ../deps/v8/src/base/platform/platform.h:472
No locals.
#7  0x000055555727ff6c in v8::base::ThreadEntry (arg=0x5555580f85e0)
    at ../deps/v8/src/base/platform/platform-posix.cc:600
        thread = 0x5555580f85e0
#8  0x00007ffff6f0e519 in start_thread (arg=0x7ffff6362700) at pthread_create.c:456
        __res = <optimized out>
        pd = 0x7ffff6362700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737324132096, 3140752450534644803, 
                140737488347454, 140737488347455, 0, 140737324132096, -3140735333997601725, 
                -3140736997479801789}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 
              0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#9  0x00007ffff6c50a4f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
No locals.

Thread 3 (Thread 0x7ffff6b63700 (LWP 12863)):
#0  0x00007ffff6f17666 in futex_abstimed_wait_cancelable (private=0, abstime=0x0, 
    expected=0, futex_word=0x5555580f80f8)
    at ../sysdeps/unix/sysv/linux/futex-internal.h:205
        __ret = -512
        oldtype = 0
        err = <optimized out>
#1  do_futex_wait (sem=sem@entry=0x5555580f80f8, abstime=0x0) at sem_waitcommon.c:111
No locals.
#2  0x00007ffff6f17724 in __new_sem_wait_slow (sem=0x5555580f80f8, abstime=0x0)
    at sem_waitcommon.c:181
        _buffer = {__routine = 0x7ffff6f17620 <__sem_wait_cleanup>, __arg = 0x5555580f80f8, 
          __canceltype = -155832848, __prev = 0x0}
        err = <optimized out>
        d = 8589934592
#3  0x000055555727d166 in v8::base::Semaphore::Wait (this=0x5555580f80f8)
    at ../deps/v8/src/base/platform/semaphore.cc:103
        result = 32767
#4  0x000055555707b41c in v8::platform::TaskQueue::GetNext (this=0x5555580f80c8)
    at ../deps/v8/src/libplatform/task-queue.cc:44
No locals.
#5  0x000055555707b590 in v8::platform::WorkerThread::Run (this=0x5555580f8430)
    at ../deps/v8/src/libplatform/worker-thread.cc:25
        task = 0x5555581388a0
#6  0x000055555727f14a in v8::base::Thread::NotifyStartedAndRun (this=0x5555580f8430)
    at ../deps/v8/src/base/platform/platform.h:472
No locals.
#7  0x000055555727ff6c in v8::base::ThreadEntry (arg=0x5555580f8430)
    at ../deps/v8/src/base/platform/platform-posix.cc:600
        thread = 0x5555580f8430
#8  0x00007ffff6f0e519 in start_thread (arg=0x7ffff6b63700) at pthread_create.c:456
        __res = <optimized out>
        pd = 0x7ffff6b63700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737332524800, 3140752450534644803, 
                140737488347454, 140737488347455, 0, 140737332524800, -3140736432972358589, 
                -3140736997479801789}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 
              0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#9  0x00007ffff6c50a4f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
No locals.

Thread 2 (Thread 0x7ffff7ff6700 (LWP 12862)):
#0  0x00007ffff6f17666 in futex_abstimed_wait_cancelable (private=0, abstime=0x0, 
    expected=0, futex_word=0x5555580dffc0 <node::debug_semaphore>)
    at ../sysdeps/unix/sysv/linux/futex-internal.h:205
       __ret = -512
        oldtype = 0
        err = <optimized out>
#1  do_futex_wait (sem=sem@entry=0x5555580dffc0 <node::debug_semaphore>, abstime=0x0)
    at sem_waitcommon.c:111
No locals.
#2  0x00007ffff6f17724 in __new_sem_wait_slow (sem=0x5555580dffc0 <node::debug_semaphore>, 
    abstime=0x0) at sem_waitcommon.c:181
        _buffer = {__routine = 0x7ffff6f17620 <__sem_wait_cleanup>, 
          __arg = 0x5555580dffc0 <node::debug_semaphore>, __canceltype = 0, __prev = 0x0}
        err = <optimized out>
        d = 0
#3  0x0000555557271911 in uv_sem_wait (sem=0x5555580dffc0 <node::debug_semaphore>)
    at ../deps/uv/src/unix/thread.c:528
        r = 0
#4  0x0000555556fd6f95 in node::DebugSignalThreadMain (unused=0x0) at ../src/node.cc:4171
No locals.
#5  0x00007ffff6f0e519 in start_thread (arg=0x7ffff7ff6700) at pthread_create.c:456
        __res = <optimized out>
        pd = 0x7ffff7ff6700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737354098432, 3140752450534644803, 
                140737488347422, 140737488347423, 0, 140737354098432, -3140734863698682813, 
                -3140736997479801789}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 
              0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#6  0x00007ffff6c50a4f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
No locals.

Thread 1 (Thread 0x7ffff7fe2740 (LWP 12858)):
#0  0x00000a68e6b7ed68 in ?? ()
No symbol table info available.
#1  0x4073f00000000000 in ?? ()
No symbol table info available.
#2  0x4033000000000000 in ?? ()
No symbol table info available.
#3  0x023b145957053b00 in ?? ()
No symbol table info available.
#4  0x000001a566f538c1 in ?? ()
No symbol table info available.
#5  0x000001a566f53d39 in ?? ()
No symbol table info available.
#6  0x00003398b0ddea41 in ?? ()
No symbol table info available.
#7  0x000001a566f74eb1 in ?? ()
No symbol table info available.
#8  0x00007fffffff9508 in ?? ()
No symbol table info available.
#9  0x00000a68e6b14d11 in ?? ()
No symbol table info available.
#10 0x0000002e00000000 in ?? ()
No symbol table info available.
#11 0x000028e0ebf50611 in ?? ()
No symbol table info available.
#12 0x0000555558138990 in ?? ()
No symbol table info available.
#13 0x00005555580f93a0 in ?? ()
No symbol table info available.
#14 0x0000000000000000 in ?? ()
No symbol table info available.

disassemble:

No function contains program counter for selected frame.

info registers:

rax            0x3dedd3a0a819   68091667064857
rbx            0x3a8181c043c1   64327902053313
rcx            0x1823e1352be9   26542381280233
rdx            0x28e0ebf50611   44946496488977
rsi            0x1a566f6e559    1809908688217
rdi            0x23b145957053b00        160744585646390016
rbp            0x7fffffff9470   0x7fffffff9470
rsp            0x7fffffff9438   0x7fffffff9438
r8             0x1823e1352bd9   26542381280217
r9             0x1a566f6e559    1809908688217
r10            0x267b44f00000   42310879412224
r11            0x3a8181c04381   64327902053249
r12            0x28e0ebf574c9   44946496517321
r13            0x5555580f9458   93825037997144
r14            0x1a566f53b31    1809908579121
r15            0x28e0ebf574f1   44946496517361
rip            0xa68e6b7ed68    0xa68e6b7ed68
eflags         0x10246  [ PF ZF IF RF ]
cs             0x33     51
ss             0x2b     43
ds             0x0      0
es             0x0      0
fs             0x0      0
gs             0x0      0

bnoordhuis commented 6 years ago

Interesting. Is the backtrace for thread 1 always the same? Does ./configure --gdb && make ... produce a more readable backtrace?

You should be able to get a disassembly with disassemble $rip-32,+64. You may need to tweak the offsets if it looks like garbage but worst case I'll piece it together from the hexadecimal codes.

simevo commented 6 years ago

as I still have the gdb session open in a screen, it's easy for me right now to paste here first the output of disassemble $rip-32,+64:

Dump of assembler code from 0xa68e6b7ed48 to 0xa68e6b7ed88:
   0x00000a68e6b7ed48:  or     %al,(%r15,%rcx,1)
   0x00000a68e6b7ed4c:  test   %ecx,%esp
   0x00000a68e6b7ed4e:  add    %eax,(%rax)
   0x00000a68e6b7ed50:  add    %ch,%cl
   0x00000a68e6b7ed52:  pop    %rsp
   0x00000a68e6b7ed53:  add    %al,(%rax)
   0x00000a68e6b7ed55:  add    %cl,-0x75(%rax)
   0x00000a68e6b7ed58:  push   %rbp
   0x00000a68e6b7ed59:  sbb    %cl,-0x48(%rax)
   0x00000a68e6b7ed5c:  sbb    %ebp,0x3dedd3a0(%rax)
   0x00000a68e6b7ed62:  add    %al,(%rax)
   0x00000a68e6b7ed64:  mov    -0x28(%rbp),%rdi
=> 0x00000a68e6b7ed68:  cmp    %rax,-0x1(%rdi)
   0x00000a68e6b7ed6c:  jne    0xa68e6b7f320
   0x00000a68e6b7ed72:  push   %rdx
   0x00000a68e6b7ed73:  pushq  -0x20(%rbp)
   0x00000a68e6b7ed76:  mov    $0x1,%eax
   0x00000a68e6b7ed7b:  movabs $0x3a8181cb0731,%rsi
   0x00000a68e6b7ed85:  callq  0xa68e620bc00
End of assembler dump.

let me know if this is relevant; in the meantime I'll do the reconfigure / make

bnoordhuis commented 6 years ago

It looks like V8 expected to see an object pointer in %rdi at the offending instruction. Would be interesting to see if other crashes follow a similar pattern.

simevo commented 6 years ago

original:

(gdb) where
#0  0x00000a68e6b7ed68 in ?? ()
#1  0x4073f00000000000 in ?? ()
#2  0x4033000000000000 in ?? ()
#3  0x023b145957053b00 in ?? ()
#4  0x000001a566f538c1 in ?? ()
#5  0x000001a566f53d39 in ?? ()
#6  0x00003398b0ddea41 in ?? ()
#7  0x000001a566f74eb1 in ?? ()
#8  0x00007fffffff9508 in ?? ()
#9  0x00000a68e6b14d11 in ?? ()
#10 0x0000002e00000000 in ?? ()
#11 0x000028e0ebf50611 in ?? ()
#12 0x0000555558138990 in ?? ()
#13 0x00005555580f93a0 in ?? ()
#14 0x0000000000000000 in ?? ()

another one:

where
#0  0x000022b9d1b6c628 in ?? ()
#1  0x4088680000000000 in ?? ()
#2  0x406d800000000000 in ?? ()
#3  0x023b145957053b00 in ?? ()
#4  0x00002388b08e7381 in ?? ()
#5  0x00002388b08e77f9 in ?? ()
#6  0x00003396d75cbeb9 in ?? ()
#7  0x00002388b088b7b1 in ?? ()
#8  0x00007fffffff91a8 in ?? ()
#9  0x000022b9d1b3706b in ?? ()
#10 0x0000002e00000000 in ?? ()
#11 0x00001d524d961611 in ?? ()
#12 0x00000000ffff9170 in ?? ()
#13 0x00005555580f93a0 in ?? ()
#14 0x00007fffffff9160 in ?? ()
#15 0x00005555569fb443 in v8::internal::(anonymous namespace)::ElementsAccessorBase<v8::internal::(anonymous namespace)::FastHoleyObjectElementsAccessor, v8::internal::(anonymous namespace)::ElementsKindTraits<(v8::internal::ElementsKind)3> >::GetImpl (holder=..., entry=13206)
    at ../deps/v8/src/elements.cc:566

disassemble $rip-32,+64
Dump of assembler code from 0x22b9d1b6c608 to 0x22b9d1b6c648:
   0x000022b9d1b6c608:  or     %al,(%r15,%rcx,1)
   0x000022b9d1b6c60c:  test   %ecx,%esp
   0x000022b9d1b6c60e:  add    %eax,(%rax)
   0x000022b9d1b6c610:  add    %ch,%cl
   0x000022b9d1b6c612:  pop    %rsp
   0x000022b9d1b6c613:  add    %al,(%rax)
   0x000022b9d1b6c615:  add    %cl,-0x75(%rax)
   0x000022b9d1b6c618:  push   %rbp
   0x000022b9d1b6c619:  sbb    %cl,-0x48(%rax)
   0x000022b9d1b6c61c:  sbb    %ebp,0x1d7f5f90(%rax)
   0x000022b9d1b6c622:  add    %al,(%rax)
   0x000022b9d1b6c624:  mov    -0x28(%rbp),%rdi
=> 0x000022b9d1b6c628:  cmp    %rax,-0x1(%rdi)
   0x000022b9d1b6c62c:  jne    0x22b9d1b6cbe0
   0x000022b9d1b6c632:  push   %rdx
   0x000022b9d1b6c633:  pushq  -0x20(%rbp)
   0x000022b9d1b6c636:  mov    $0x1,%eax
   0x000022b9d1b6c63b:  movabs $0x24fb46cb0731,%rsi
   0x000022b9d1b6c645:  callq  0x22b9d120bc00
End of assembler dump.

yet another one:

(gdb) where
#0  0x000026fecec46948 in ?? ()
#1  0x4088680000000000 in ?? ()
#2  0x406d800000000000 in ?? ()
#3  0x023b145957053b00 in ?? ()
#4  0x00002ba0b0b52119 in ?? ()
#5  0x00002ba0b0b52591 in ?? ()
#6  0x000011b9d53a09e9 in ?? ()
#7  0x00002ba0b0b2ed61 in ?? ()
#8  0x00007fffffff91a8 in ?? ()
#9  0x000026fecec0ff2b in ?? ()
#10 0x0000002e00000000 in ?? ()
#11 0x000025635a6d5e21 in ?? ()
#12 0x00000000ffff9170 in ?? ()
#13 0x00005555580f93a0 in ?? ()
#14 0x00007fffffff9160 in ?? ()
#15 0x00005555569fb443 in v8::internal::(anonymous namespace)::ElementsAccessorBase<v8::internal::(anonymous namespace)::FastHoleyObjectElementsAccessor, v8::internal::(anonymous namespace)::ElementsKindTraits<(v8::internal::ElementsKind)3> >::GetImpl (holder=..., entry=4537)
    at ../deps/v8/src/elements.cc:566
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

disassemble $rip-32,+64
Dump of assembler code from 0x26fecec46928 to 0x26fecec46968:
   0x000026fecec46928:  or     %al,(%r15,%rcx,1)
   0x000026fecec4692c:  test   %ecx,%esp
   0x000026fecec4692e:  add    %eax,(%rax)
   0x000026fecec46930:  add    %ch,%cl
   0x000026fecec46932:  pop    %rsp
   0x000026fecec46933:  add    %al,(%rax)
   0x000026fecec46935:  add    %cl,-0x75(%rax)
   0x000026fecec46938:  push   %rbp
   0x000026fecec46939:  sbb    %cl,-0x48(%rax)
   0x000026fecec4693c:  sbb    %ebp,0x8ba12a0(%rax)
   0x000026fecec46942:  add    %al,(%rax)
   0x000026fecec46944:  mov    -0x28(%rbp),%rdi
=> 0x000026fecec46948:  cmp    %rax,-0x1(%rdi)
   0x000026fecec4694c:  jne    0x26fecec46f00
   0x000026fecec46952:  push   %rdx
   0x000026fecec46953:  pushq  -0x20(%rbp)
   0x000026fecec46956:  mov    $0x1,%eax
   0x000026fecec4695b:  movabs $0x6fc0bcb0731,%rsi
   0x000026fecec46965:  callq  0x26fece30bc00
End of assembler dump.

in the meantime the reconfig is done so this last one is with node configured with --gdb:

#0  0x000038c381b42488 in ?? ()
#1  0x4088680000000000 in ?? ()
#2  0x406d800000000000 in ?? ()
#3  0x023b145957053b00 in ?? ()
#4  0x000027869b78a161 in ?? ()
#5  0x000027869b78a5d9 in ?? ()
#6  0x00003608992dfd39 in ?? ()
#7  0x000027869b7678b9 in ?? ()
#8  0x00007fffffff91a8 in ?? ()
#9  0x000038c381b0b411 in ?? ()
#10 0x0000002e00000000 in ?? ()
#11 0x000033ebeecf1b41 in ?? ()
#12 0x00000000ffff9170 in ?? ()
#13 0x000055555810c3a0 in ?? ()
#14 0x00007fffffff9160 in ?? ()
#15 0x0000555556a03661 in v8::internal::(anonymous namespace)::ElementsAccessorBase<v8::internal::(anonymous namespace)::FastHoleyObjectElementsAccessor, v8::internal::(anonymous namespace)::ElementsKindTraits<(v8::internal::ElementsKind)3> >::GetImpl (holder=..., entry=13832)
    at ../deps/v8/src/elements.cc:566
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

simevo commented 6 years ago

Hi It was interesting to see how one investigates a node segfault but from out POV this has now a considerably lower priority.

Turned out debian will go from node v4 (stable) straight to v8 (newstable); node v6 will never exist on debian, and this issue is not reproducible in v8.

I'll close this bug, feel free to reopen if you need. Thanks again for the prompt support !

nodejs / node

v6.12.x segfault on linux #17680