search

facebook / hhvm

A virtual machine for executing programs written in Hack.
https://hhvm.com
Other
18.19k stars 3k forks source link

Segmentation fault during retranslation #7680

Open Norbyte opened 7 years ago

Norbyte commented 7 years ago

HHVM Version

HipHop VM 3.18.0-dev (rel) Compiler: heads/master-0-g3fb5e0b2512234155bb07fd7bfa18b4047c42deb Repo schema: 3ddd9660d9a7ad9cd9acd0503620d1eb9269e95a

Standalone code, or other way to reproduce the problem

HHVM crashes every time it reaches the request count specified in the config variable JitRetranslateAllRequest (default 3000). Disabling PGO using hhvm.jit_pgo = false or retranslation using -vEval.JitRetranslateAllRequest=0 is a viable workaround. Setting a very low JitRetranslateAllRequest value (around 200-300) seems to reduce the probability of crashing.

This was tested on an internal application, using the ab -c 10 -n 10000 http://<url> benchmark. I'm not sure if this is reproducible using other codebases.

HHVM Output

Finished singleJitRequest 1
Finished singleJitRequest 2
[...]
Finished singleJitRequest 23
Scheduling the retranslation of all profiled translations
retranslateAll: starting to build the call graph
retranslateAll: finished building the call graph
Segmentation fault

Backtrace

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffa77ff700 (LWP 7196)]
0x0000000001f89b51 in HPHP::jit::tc::emitTranslation(HPHP::jit::TransEnv, folly::Optional<HPHP::jit::CodeCache::View>) ()
Missing separate debuginfos, [...]
(gdb) bt
#0  0x0000000001f89b51 in HPHP::jit::tc::emitTranslation(HPHP::jit::TransEnv, folly::Optional<HPHP::jit::CodeCache::View>) ()
#1  0x0000000001da2fc2 in HPHP::jit::mcgen::translate(HPHP::jit::TransArgs, HPHP::jit::FPInvOffset, folly::Optional<HPHP::jit::CodeCache::View>) ()
#2  0x0000000001d96fa6 in HPHP::jit::mcgen::(anonymous namespace)::regeneratePrologue(int, HPHP::jit::tc::FuncMetaInfo&)::{lambda(bool)#1}::operator()(bool) const ()
#3  0x0000000001d9bf8b in HPHP::jit::mcgen::regeneratePrologues(HPHP::Func*, HPHP::jit::tc::FuncMetaInfo&) ()
#4  0x0000000001db2a61 in HPHP::jit::mcgen::(anonymous namespace)::optimize(HPHP::Func*, unsigned char*) ()
#5  0x0000000001db923e in HPHP::jit::mcgen::(anonymous namespace)::TranslateWorker::doJob(HPHP::jit::mcgen::(anonymous namespace)::OptimizeData*) ()
#6  0x0000000001d9d00d in HPHP::JobQueueWorker<HPHP::jit::mcgen::(anonymous namespace)::OptimizeData*, void*, true, true, HPHP::detail::NoDropCachePolicy>::start() ()
#7  0x0000000000d9b3b7 in HPHP::AsyncFuncImpl::ThreadFunc(void*) ()
#8  0x000000000142ae97 in HPHP::start_routine_wrapper(void*) ()
#9  0x00007ffff559fdc5 in start_thread () from /lib64/libpthread.so.0
#10 0x00007fffefa7673d in clone () from /lib64/libc.so.6
Orvid commented 7 years ago

@paulbiss I don't think this is supposed to happen

paulbiss commented 7 years ago

I'm not sure what's happening here, can you try this with a debug build or see if there's anything else you can learn about the top frame in your core? It would be helpful to know precisely where the segmentation fault is.

Norbyte commented 7 years ago

I re-tested this using a debug build. Still crashes, but with a different stack trace this time:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffbabff700 (LWP 13692)]
HPHP::Func::getFuncId (this=0x7fffbabfa6b0) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/vm/func-inl.h:87
87      inline FuncId Func::getFuncId() const {

(gdb) bt
#0  HPHP::Func::getFuncId (this=0x7fffbabfa6b0) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/vm/func-inl.h:87
#1  0x0000000005e14e5d in HPHP::jit::tc::FuncMetaInfo::FuncMetaInfo(HPHP::Func*, HPHP::jit::tc::ThreadTCBuffer&&) (this=0x7fffbabfaa30, f=0x7fffe7aa6100,
    buf=<unknown type in /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/hhvm/hhvm, CU 0x1d3b90eb, DIE 0x1d57892b>) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/vm/jit/tc.h:97
#2  0x0000000005e155e7 in HPHP::jit::mcgen::(anonymous namespace)::optimize (func=0x7fffe7aa6100, localBuf=0x4100000 "nEEESt10shared_ptrIT_ERKT0_DpOT1_") at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/vm/jit/mcgen-translate.cpp:56
#3  0x0000000005e15d82 in HPHP::jit::mcgen::(anonymous namespace)::TranslateWorker::doJob (this=0x7fffd77f1cd0, d=0x7fffd738bb60) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/vm/jit/mcgen-translate.cpp:114
#4  0x0000000005e1e9aa in HPHP::JobQueueWorker<HPHP::jit::mcgen::(anonymous namespace)::OptimizeData*, void*, true, true, HPHP::detail::NoDropCachePolicy>::start (this=0x7fffd77f1cd0)
    at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/util/job-queue.h:463
#5  0x00000000045e39d6 in HPHP::AsyncFunc<HPHP::AsyncFileCacheSaver>::run_ (obj=0x7fffd73ed000) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/util/async-func.h:203
#6  0x000000000489cd2f in HPHP::AsyncFuncImpl::threadFuncImpl (this=0x7fffd73ed000) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/util/async-func.cpp:139
#7  0x000000000489c901 in HPHP::AsyncFuncImpl::ThreadFunc (obj=0x7fffd73ed000) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/util/async-func.cpp:57
#8  0x0000000004ec6691 in HPHP::start_routine_wrapper (arg=0x7fffd7b16b00) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/base/thread-hooks.cpp:105
#9  0x00007ffff559fdc5 in start_thread () from /lib64/libpthread.so.0
#10 0x00007fffefa7673d in clone () from /lib64/libc.so.6

(gdb) print *this
$1 = {static kSmallDeltaLimit = 65535, static kBitsPerQword = 64, static s___call = 0x7fffe78082c0, static s___callStatic = 0x7fffe7806d30, static kMagic = -1168197103, static s_treadmill = {<std::atomic_bool> = {_M_base = {
        _M_i = false}}, <No data fields>}, static s_totalClonedClosures = <optimized out>, m_magic = 68157440, m_funcBody = {m_s = {<std::__atomic_base<unsigned long>> = {_M_i = 68157440}, <No data fields>}}, m_cachedFunc = {
    m_handle = {<std::__atomic_base<unsigned int>> = {_M_i = 320864256}, <No data fields>}}, m_funcId = 0, m_fullName = {m_s = 320864256}, m_name = {m_s = 6291456}, m_baseCls = {m_s = 140736799126216}, m_cls = {
    m_s = {<std::__atomic_base<unsigned long>> = {_M_i = 0}, <No data fields>}}, {m_methodSlot = 0, m_namedEntity = 0}, m_maybeIntercepted = 0 '\000', m_cloned = {flag = {<std::__atomic_flag_base> = {_M_i = false}, <No data fields>}},
  m_isPreFunc = false, m_hasPrivateAncestor = false, m_shouldSampleJit = false, m_maxStackCells = 0, m_refBitVal = 140736833869088, m_unit = 0xb, m_shared = {m_px = 0x0}, m_paramCounts = 0, m_attrs = {m_attrs = {
      _M_i = HPHP::AttrNone}}, m_prologueTable = {{m_s = {<std::__atomic_base<unsigned long>> = {_M_i = 1065353216}, <No data fields>}}}}

(gdb) x/i $pc
=> 0x4e19ec8 <HPHP::Func::getFuncId() const>:   push   %rbp

Dump of other threads:

(gdb) info thread
  Id   Target Id         Frame
* 12   Thread 0x7fffbabff700 (LWP 13692) "hhvm" HPHP::Func::getFuncId (this=0x7fffbabfa6b0) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/vm/func-inl.h:87
  11   Thread 0x7fffbb3ff700 (LWP 13691) "hhvm" std::allocator<std::string>::allocator (this=0x7fffd916ac98) at /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/bits/allocator.h:121
  10   Thread 0x7fffbbfff700 (LWP 13690) "hhvm" std::_Rb_tree<std::string, std::string, std::_Identity<std::string>, std::less<std::string>, std::allocator<std::string> >::_M_begin (this=0x7fffbbffad48)
    at /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/bits/stl_tree.h:528
  9    Thread 0x7fffc0bff700 (LWP 13689) "hhvm" 0x0000000004b44681 in HPHP::HeapObject::initHeader (this=0x7fffdac91c00, kind=HPHP::Hole, count=3584) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/base/header-kind.h:113
  8    Thread 0x7fffd9bff700 (LWP 13688) "hhvm" 0x00007ffff55a36d5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  7    Thread 0x7fffda3ff700 (LWP 13687) "hhvm" memcpy () at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/util/memcpy-x64.S:123
  6    Thread 0x7fffdabff700 (LWP 13686) "hhvm" 0x0000000004b44681 in HPHP::HeapObject::initHeader (this=0x7fffd8ca0000, kind=HPHP::Hole, count=6144) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/base/header-kind.h:113
  5    Thread 0x7fffd57ff700 (LWP 13683) "hhvm" 0x0000000004b44681 in HPHP::HeapObject::initHeader (this=0x7fffb1c88fb0, kind=HPHP::Hole, count=112) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/base/header-kind.h:113
  4    Thread 0x7fffd63fe700 (LWP 13681) "hhvm" 0x00007fffefa76d13 in epoll_wait () from /lib64/libc.so.6
  3    Thread 0x7fffd6bff700 (LWP 13680) "hhvm" 0x00007fffefa76d13 in epoll_wait () from /lib64/libc.so.6
  1    Thread 0x7ffff7fbd7c0 (LWP 13671) "hhvm" 0x00007ffff55a36d5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0

(gdb) thread 11
[Switching to thread 11 (Thread 0x7fffbb3ff700 (LWP 13691))]
#0  std::allocator<std::string>::allocator (this=0x7fffd916ac98) at /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/bits/allocator.h:121
121           ~allocator() throw() { }
(gdb) bt
#0  std::allocator<std::string>::allocator (this=0x7fffd916ac98) at /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/bits/allocator.h:121
#1  0x00000000061ea401 in HPHP::jit::(anonymous namespace)::RegState::RegState (this=0x7fffd916ac98) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/vm/jit/vasm-copy.cpp:98
#2  0x00000000061ea4fb in std::_Construct<HPHP::jit::(anonymous namespace)::RegState> (__p=0x7fffd916ac98) at /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/bits/stl_construct.h:75
#3  0x00000000061ea16a in std::__uninitialized_default_n_1<false>::__uninit_default_n<HPHP::jit::(anonymous namespace)::RegState*, unsigned long> (__first=0x7fffd915b000, __n=73)
    at /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/bits/stl_uninitialized.h:495
#4  0x00000000061e91ba in std::__uninitialized_default_n<HPHP::jit::(anonymous namespace)::RegState*, unsigned long> (__first=0x7fffd915b000, __n=132)
    at /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/bits/stl_uninitialized.h:543
#5  0x00000000061e551b in std::__uninitialized_default_n_a<HPHP::jit::(anonymous namespace)::RegState*, unsigned long, HPHP::jit::(anonymous namespace)::RegState> (__first=0x7fffd915b000, __n=132)
    at /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/bits/stl_uninitialized.h:605
#6  0x00000000061dae42 in std::vector<HPHP::jit::(anonymous namespace)::RegState, std::allocator<HPHP::jit::(anonymous namespace)::RegState> >::_M_default_append (this=0x7fffbb3f9c30, __n=132)
    at /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/bits/vector.tcc:557
#7  0x00000000061d3581 in std::vector<HPHP::jit::(anonymous namespace)::RegState, std::allocator<HPHP::jit::(anonymous namespace)::RegState> >::resize (this=0x7fffbb3f9c30, __new_size=132)
    at /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/bits/stl_vector.h:667
#8  0x00000000061cfcde in HPHP::jit::(anonymous namespace)::initialize_reg_states (env=...) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/vm/jit/vasm-copy.cpp:200
#9  0x00000000061d215c in HPHP::jit::(anonymous namespace)::analyze_defs (env=...) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/vm/jit/vasm-copy.cpp:565
#10 0x00000000061d3325 in HPHP::jit::optimizeCopies (unit=..., abi=...) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/vm/jit/vasm-copy.cpp:850
#11 0x00000000063e8e80 in HPHP::jit::optimizeX64 (unit=..., abi=..., regalloc=true) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/vm/jit/vasm-x64.cpp:1063
#12 0x0000000005dd775b in HPHP::jit::irlower::(anonymous namespace)::optimize (unit=..., kind=HPHP::jit::Trace, regAlloc=true) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/vm/jit/irlower.cpp:102
#13 0x0000000005dd8178 in HPHP::jit::irlower::lowerUnit (unit=..., kind=HPHP::jit::Trace, regAlloc=true) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/vm/jit/irlower.cpp:166
#14 0x0000000005e17b18 in HPHP::jit::mcgen::translate (args=..., spOff=..., optView=...) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/vm/jit/mcgen-translate.cpp:402
#15 0x0000000005e1588f in HPHP::jit::mcgen::(anonymous namespace)::optimize (func=0x7fffe7859420, localBuf=0x2080000 "\200", <incomplete sequence \350>) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/vm/jit/mcgen-translate.cpp:79
#16 0x0000000005e15d82 in HPHP::jit::mcgen::(anonymous namespace)::TranslateWorker::doJob (this=0x7fffd77f1c40, d=0x7fffd738b5b0) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/vm/jit/mcgen-translate.cpp:114
#17 0x0000000005e1e9aa in HPHP::JobQueueWorker<HPHP::jit::mcgen::(anonymous namespace)::OptimizeData*, void*, true, true, HPHP::detail::NoDropCachePolicy>::start (this=0x7fffd77f1c40)
    at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/util/job-queue.h:463
#18 0x00000000045e39d6 in HPHP::AsyncFunc<HPHP::AsyncFileCacheSaver>::run_ (obj=0x7fffd73ebb00) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/util/async-func.h:203
#19 0x000000000489cd2f in HPHP::AsyncFuncImpl::threadFuncImpl (this=0x7fffd73ebb00) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/util/async-func.cpp:139
#20 0x000000000489c901 in HPHP::AsyncFuncImpl::ThreadFunc (obj=0x7fffd73ebb00) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/util/async-func.cpp:57
#21 0x0000000004ec6691 in HPHP::start_routine_wrapper (arg=0x7fffd7b16a20) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/base/thread-hooks.cpp:105
#22 0x00007ffff559fdc5 in start_thread () from /lib64/libpthread.so.0
#23 0x00007fffefa7673d in clone () from /lib64/libc.so.6

(gdb) thread 10
[Switching to thread 10 (Thread 0x7fffbbfff700 (LWP 13690))]
#0  std::_Rb_tree<std::string, std::string, std::_Identity<std::string>, std::less<std::string>, std::allocator<std::string> >::_M_begin (this=0x7fffbbffad48)
    at /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/bits/stl_tree.h:528
528           _S_left(_Base_ptr __x)
(gdb) bt
#0  std::_Rb_tree<std::string, std::string, std::_Identity<std::string>, std::less<std::string>, std::allocator<std::string> >::_M_begin (this=0x7fffbbffad48)
    at /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/bits/stl_tree.h:528
#1  0x0000000005564382 in std::_Rb_tree<unsigned int, std::pair<unsigned int const, std::unordered_set<unsigned int, std::hash<unsigned int>, std::equal_to<unsigned int>, std::allocator<unsigned int> > >, std::_Select1st<std::pair<unsigned int const, std::unordered_set<unsigned int, std::hash<unsigned int>, std::equal_to<unsigned int>, std::allocator<unsigned int> > > >, std::less<unsigned int>, std::allocator<std::pair<unsigned int const, std::unordered_set<unsigned int, std::hash<unsigned int>, std::equal_to<unsigned int>, std::allocator<unsigned int> > > > >::~_Rb_tree (this=0x7fffbbffad18, __in_chrg=<optimized out>)
    at /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/bits/stl_tree.h:671
#2  0x0000000005563394 in std::map<unsigned int, std::unordered_set<unsigned int, std::hash<unsigned int>, std::equal_to<unsigned int>, std::allocator<unsigned int> >, std::less<unsigned int>, std::allocator<std::pair<unsigned int const, std::unordered_set<unsigned int, std::hash<unsigned int>, std::equal_to<unsigned int>, std::allocator<unsigned int> > > > >::~map (this=0x7fffbbffad18, __in_chrg=<optimized out>)
    at /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/bits/stl_map.h:96
#3  0x00000000055633b2 in HPHP::DataBlock::~DataBlock (this=0x7fffbbffaca8, __in_chrg=<optimized out>) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/util/data-block.h:68
#4  0x000000000605f2a5 in HPHP::jit::tc::ThreadTCBuffer::~ThreadTCBuffer (this=0x7fffbbffaac0, __in_chrg=<optimized out>) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/vm/jit/tc-internal.cpp:340
#5  0x0000000005e15466 in HPHP::jit::tc::FuncMetaInfo::~FuncMetaInfo (this=0x7fffbbffaab0, __in_chrg=<optimized out>) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/vm/jit/tc.h:94
#6  0x0000000005e16f68 in HPHP::jit::mcgen::(anonymous namespace)::OptimizeData::~OptimizeData (this=0x7fffbbffaaa0, __in_chrg=<optimized out>) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/vm/jit/mcgen-translate.cpp:89
#7  0x0000000005e17130 in HPHP::jit::mcgen::(anonymous namespace)::retranslateAll () at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/vm/jit/mcgen-translate.cpp:289
#8  0x0000000005e18ad9 in HPHP::jit::mcgen::__lambda127::__lambda128::operator() (__closure=0x7fffd76efaf0) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/vm/jit/mcgen-translate.cpp:553
#9  0x0000000005e2b0ea in std::_Bind_simple<HPHP::jit::mcgen::checkRetranslateAll()::__lambda127::__lambda128()>::_M_invoke<>(std::_Index_tuple<>) (this=0x7fffd76efaf0)
    at /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/functional:1732
#10 0x0000000005e2b065 in std::_Bind_simple<HPHP::jit::mcgen::checkRetranslateAll()::__lambda127::__lambda128()>::operator()(void) (this=0x7fffd76efaf0)
    at /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/functional:1720
#11 0x0000000005e2ae1e in std::thread::_Impl<std::_Bind_simple<HPHP::jit::mcgen::checkRetranslateAll()::__lambda127::__lambda128()> >::_M_run(void) (this=0x7fffd76efad8)
    at /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/thread:115
#12 0x00007ffff030d230 in ?? () from /lib64/libstdc++.so.6
#13 0x00007ffff559fdc5 in start_thread () from /lib64/libpthread.so.0
#14 0x00007fffefa7673d in clone () from /lib64/libc.so.6

(gdb) thread 9
[Switching to thread 9 (Thread 0x7fffc0bff700 (LWP 13689))]
#0  0x0000000004b44681 in HPHP::HeapObject::initHeader (this=0x7fffdac91c00, kind=HPHP::Hole, count=3584) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/base/header-kind.h:113
113                   uint32_t(count) << (8 * offsetof(HeapObject, m_count));
(gdb) bt
#0  0x0000000004b44681 in HPHP::HeapObject::initHeader (this=0x7fffdac91c00, kind=HPHP::Hole, count=3584) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/base/header-kind.h:113
#1  0x0000000004d67dff in HPHP::FreeNode::InitFrom (addr=0x7fffdac91c00, size=3584, kind=HPHP::Hole) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/base/memory-manager-inl.h:117
#2  0x0000000004d6a41c in HPHP::MemoryManager::splitTail (this=0x7fffc0bfb8e0, tail=0x7fffdac91c00, tailBytes=10752, nSplit=3, splitUsable=3584, splitInd=26) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/base/memory-manager.cpp:769
#3  0x0000000004d6ab56 in HPHP::MemoryManager::slabAlloc (this=0x7fffc0bfb8e0, bytes=3288, index=26) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/base/memory-manager.cpp:842
#4  0x0000000004d6af88 in HPHP::MemoryManager::mallocSmallSizeSlow (this=0x7fffc0bfb8e0, bytes=3288, index=26) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/base/memory-manager.cpp:876
#5  0x00000000048598e3 in HPHP::MemoryManager::mallocSmallIndex (this=0x7fffc0bfb8e0, index=26, bytes=3288) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/base/memory-manager-inl.h:234
#6  0x0000000004859a26 in HPHP::MemoryManager::mallocSmallSize (this=0x7fffc0bfb8e0, bytes=3288) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/base/memory-manager-inl.h:245
#7  0x0000000004d6f442 in HPHP::req::allocate<false> (nbytes=3272, ty=0) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/base/memory-manager.cpp:952
#8  0x0000000004d6b260 in HPHP::req::malloc (nbytes=3272, tyindex=0) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/base/memory-manager.cpp:964
#9  0x0000000004cf75d8 in HPHP::req::Allocator<std::__detail::_Hash_node_base*, HPHP::type_scan::Action::Conservative<std::pair<HPHP::StringData const* const, HPHP::ExecutionContext::FileInfo> > >::allocate (this=0x7fffc0bf9317,
    num=409) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/base/req-malloc.h:147
#10 0x0000000004cf66fb in std::_Hashtable<HPHP::StringData const*, std::pair<HPHP::StringData const* const, HPHP::ExecutionContext::FileInfo>, HPHP::req::Allocator<std::pair<HPHP::StringData const* const, HPHP::ExecutionContext::FileInfo>, HPHP::type_scan::Action::Conservative<std::pair<HPHP::StringData const* const, HPHP::ExecutionContext::FileInfo> > >, std::__detail::_Select1st, HPHP::string_data_same, HPHP::string_data_hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_allocate_buckets (this=0x7fffdac061f0, __n=409)
    at /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/bits/hashtable.h:779
#11 0x0000000004f3a418 in std::_Hashtable<HPHP::StringData const*, std::pair<HPHP::StringData const* const, HPHP::ExecutionContext::FileInfo>, HPHP::req::Allocator<std::pair<HPHP::StringData const* const, HPHP::ExecutionContext::FileInfo>, HPHP::type_scan::Action::Conservative<std::pair<HPHP::StringData const* const, HPHP::ExecutionContext::FileInfo> > >, std::__detail::_Select1st, HPHP::string_data_same, HPHP::string_data_hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_rehash_aux (this=0x7fffdac061f0, __n=409)
    at /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/bits/hashtable.h:1701
#12 0x0000000004f39d7b in std::_Hashtable<HPHP::StringData const*, std::pair<HPHP::StringData const* const, HPHP::ExecutionContext::FileInfo>, HPHP::req::Allocator<std::pair<HPHP::StringData const* const, HPHP::ExecutionContext::FileInfo>, HPHP::type_scan::Action::Conservative<std::pair<HPHP::StringData const* const, HPHP::ExecutionContext::FileInfo> > >, std::__detail::_Select1st, HPHP::string_data_same, HPHP::string_data_hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_rehash (this=0x7fffdac061f0, __n=409, __state=@0x7fffc0bf9428: 199)
    at /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/bits/hashtable.h:1680
#13 0x0000000004f3933b in std::_Hashtable<HPHP::StringData const*, std::pair<HPHP::StringData const* const, HPHP::ExecutionContext::FileInfo>, HPHP::req::Allocator<std::pair<HPHP::StringData const* const, HPHP::ExecutionContext::FileInfo>, HPHP::type_scan::Action::Conservative<std::pair<HPHP::StringData const* const, HPHP::ExecutionContext::FileInfo> > >, std::__detail::_Select1st, HPHP::string_data_same, HPHP::string_data_hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_insert_unique_node (this=0x7fffdac061f0, __bkt=20, __code=9738881, __node=0x7fffdac8af50)
    at /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/bits/hashtable.h:1335
#14 0x0000000004f382ea in std::__detail::_Map_base<HPHP::StringData const*, std::pair<HPHP::StringData const* const, HPHP::ExecutionContext::FileInfo>, HPHP::req::Allocator<std::pair<HPHP::StringData const* const, HPHP::ExecutionContext::FileInfo>, HPHP::type_scan::Action::Conservative<std::pair<HPHP::StringData const* const, HPHP::ExecutionContext::FileInfo> > >, std::__detail::_Select1st, HPHP::string_data_same, HPHP::string_data_hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true>, true>::operator[](HPHP::StringData const*&&) (this=0x7fffdac061f0,
    __k=<unknown type in /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/hhvm/hhvm, CU 0xb2c01dd, DIE 0xb3e73d4>) at /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/bits/hashtable_policy.h:522
#15 0x0000000004f37960 in std::unordered_map<HPHP::StringData const*, HPHP::ExecutionContext::FileInfo, HPHP::string_data_hash, HPHP::string_data_same, HPHP::req::Allocator<std::pair<HPHP::StringData const* const, HPHP::ExecutionContext::FileInfo>, HPHP::type_scan::Action::Conservative<std::pair<HPHP::StringData const* const, HPHP::ExecutionContext::FileInfo> > > >::operator[](HPHP::StringData const*&&) (this=0x7fffdac061f0,
    __k=<unknown type in /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/hhvm/hhvm, CU 0xb2c01dd, DIE 0xb3e11c2>) at /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/bits/unordered_map.h:600
#16 0x0000000004f36d31 in HPHP::lookupUnit (path=0x7fffdac893a0, currentDir=0x7fffd732cd48 "/home/norbyte/work/www/site/new_engine/application", initial_opt=0x7fffc0bf970f)
    at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/base/unit-cache.cpp:535
#17 0x00000000051e8236 in HPHP::__lambda205::operator() (__closure=0x7fffc0bf9730) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/vm/bytecode.cpp:5165
#18 0x00000000051e84e7 in HPHP::inclOp (pc=@0x7fffc0bf9888: 0x7fffdaeba45b "\372\066", flags=HPHP::Default, opName=0x7a46737 "include") at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/vm/bytecode.cpp:5166
#19 0x00000000051e88f2 in HPHP::iopIncl (pc=@0x7fffc0bf9888: 0x7fffdaeba45b "\372\066") at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/vm/bytecode.cpp:5188
#20 0x00000000051f2464 in HPHP::iopWrapper (fn=0x51e88d0 <HPHP::iopIncl(unsigned char const*&)>, pc=@0x7fffc0bf9888: 0x7fffdaeba45b "\372\066") at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/vm/bytecode.cpp:6226
#21 0x00000000052318a0 in HPHP::interpOneIncl (fp=0x7fffd777fdf0, sp=0x7fffd777fcb0, pcOff=1114) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/vm/bytecode.cpp:6782
#22 0x0000000011a0061b in ?? ()
#23 0x0000000009e0004f in ?? ()
#24 0x00007fffbc401008 in ?? ()
#25 0x00007fffc0bf9970 in ?? ()
#26 0x000000000614b260 in HPHP::jit::enterTCImpl (start=0x7a7a7a7a7a7a7a00 <Address 0x7a7a7a7a7a7a7a00 out of bounds>, stashedAR=0x7fffdac50020) at /home/norbyte/hhvm/hhvm-3.18/hhvm/hphp/runtime/vm/jit/unique-stubs.cpp:1421
#27 0x0000000011a000c0 in ?? ()
#28 0x00007fffe221de00 in ?? ()
#29 0x00000000000000a7 in ?? ()
#30 0x00007fffe20e0ac9 in ?? ()
#31 0x00007fffdac59110 in ?? ()
#32 0x7a7a7a7a7a7a7a7a in ?? ()
#33 0x7a7a7a7a7a7a7a7a in ?? ()
#34 0x7a7a7a7a7a7a7a7a in ?? ()
#35 0x7a7a7a7a7a7a7a7a in ?? ()
#36 0x00007fffd777fea0 in ?? ()
#37 0x0000000011a000c0 in ?? ()
#38 0x00007fffe2173f00 in ?? ()
mofarrell commented 7 years ago

Are you still able to reproduce this on master? https://github.com/facebook/hhvm/commit/0b6f9e904c5429d939d63aca871799e88b844786 may have fixed this.

Norbyte commented 7 years ago

Hi,

Yes, it still crashes. Trace is exactly the same too.

Compiler: heads/master-0-g035b57723432e27d258e0cbd44a721752d7f22ad Repo schema: 844975292aae4b150a16caa221acfb3e9e48809b

Program received signal SIGSEGV, Segmentation fault.
[...]
#0  0x0000000001fb8ad1 in HPHP::jit::tc::emitTranslation(HPHP::jit::TransEnv, folly::Optional<HPHP::jit::CodeCache::View>) ()
#1  0x0000000001db4ba2 in HPHP::jit::mcgen::translate(HPHP::jit::TransArgs, HPHP::jit::FPInvOffset, folly::Optional<HPHP::jit::CodeCache::View>) ()
#2  0x0000000001da9046 in HPHP::jit::mcgen::(anonymous namespace)::regeneratePrologue(int, HPHP::jit::tc::FuncMetaInfo&)::{lambda(bool)#1}::operator()(bool) const ()
#3  0x0000000001dadebe in HPHP::jit::mcgen::regeneratePrologues(HPHP::Func*, HPHP::jit::tc::FuncMetaInfo&) ()
#4  0x0000000001dc2bbf in HPHP::jit::mcgen::(anonymous namespace)::optimize(HPHP::Func*, unsigned char*) ()
#5  0x0000000001dc9462 in HPHP::jit::mcgen::(anonymous namespace)::TranslateWorker::doJob(HPHP::jit::mcgen::(anonymous namespace)::OptimizeData*) ()
#6  0x0000000001daef4d in HPHP::JobQueueWorker<HPHP::jit::mcgen::(anonymous namespace)::OptimizeData*, void*, true, true, HPHP::detail::NoDropCachePolicy>::start() ()
#7  0x0000000000d80437 in HPHP::AsyncFuncImpl::ThreadFunc(void*) ()
#8  0x0000000001418637 in HPHP::start_routine_wrapper(void*) ()
#9  0x00007ffff5e2edc5 in start_thread () from /lib64/libpthread.so.0
#10 0x00007fffefa7673d in clone () from /lib64/libc.so.6

I can try to bisect the commit that broke it, if it helps, but it may be some time before its done, as hhvm compilation is quite slow.

mofarrell commented 7 years ago

@Norbyte my guess is bisecting this won't help that much. It probably was added with the rest of the retranslateAll code. https://github.com/facebook/hhvm/commit/3bd19f4b762cae40d35584f900e957ad3265f379 https://github.com/facebook/hhvm/commit/6d5f0f7520d5bc29c9b635de1d102675365af834

Really any of these too. https://github.com/facebook/hhvm/search?q=retranslateAll&type=Commits

This was the commit that enabled it: https://github.com/facebook/hhvm/commit/eb18082d00efd2f63ecca4d8651fd0725378e49f