jclulow
commented
2 weeks ago Related notes on the issue with building it as a shared library:
$ PATH=/usr/gnu/bin:$PATH gmake -j8
==== Building LuaJIT 2.1 ====
gmake -C src
gmake[1]: Entering directory '/ws/safari/luajit/src'
DYNLINK libluajit.so
Text relocation remains referenced
against symbol offset in file
lj_tab_len 0x63c lj_vm_dyn.o
lj_meta_cat 0xa71 lj_vm_dyn.o
lj_func_closeuv 0xd15 lj_vm_dyn.o
lj_gc_barrieruv 0xc20 lj_vm_dyn.o
lj_gc_barrieruv 0xc88 lj_vm_dyn.o
lj_tab_newkey 0x120d lj_vm_dyn.o
lj_tab_dup 0xe28 lj_vm_dyn.o
lj_gc_step_fixtop 0xdf5 lj_vm_dyn.o
lj_gc_step_fixtop 0xe5c lj_vm_dyn.o
lj_tab_new 0xdba lj_vm_dyn.o
lj_func_newL_gc 0xd55 lj_vm_dyn.o
lj_state_growstack 0x16c8 lj_vm_dyn.o
lj_state_growstack 0x1e31 lj_vm_dyn.o
lj_state_growstack 0x1eed lj_vm_dyn.o
lj_state_growstack 0x2bd4 lj_vm_dyn.o
lj_state_growstack 0x2d17 lj_vm_dyn.o
lj_state_growstack 0x39b9 lj_vm_dyn.o
lj_tab_reasize 0x133a lj_vm_dyn.o
lj_err_throw 0x1e6b lj_vm_dyn.o
lj_tab_setinth 0x2314 lj_vm_dyn.o
lj_meta_tset 0x22a1 lj_vm_dyn.o
lj_tab_getinth 0x2211 lj_vm_dyn.o
lj_tab_getinth 0x294e lj_vm_dyn.o
lj_meta_tget 0x21b8 lj_vm_dyn.o
lj_meta_istype 0x2405 lj_vm_dyn.o
lj_meta_equal_cd 0x23e6 lj_vm_dyn.o
lj_meta_equal 0x23c7 lj_vm_dyn.o
lj_meta_comp 0x234a lj_vm_dyn.o
lj_meta_for 0x2525 lj_vm_dyn.o
lj_meta_call 0x24ca lj_vm_dyn.o
lj_meta_len 0x249d lj_vm_dyn.o
lj_meta_arith 0x245a lj_vm_dyn.o
lj_tab_next 0x2831 lj_vm_dyn.o
lj_strfmt_num 0x27e3 lj_vm_dyn.o
lj_tab_get 0x273a lj_vm_dyn.o
lj_ffh_coroutine_wrap_err 0x2d04 lj_vm_dyn.o
lj_buf_putstr_lower 0x34d1 lj_vm_dyn.o
lj_buf_tostr 0x346d lj_vm_dyn.o
lj_buf_tostr 0x34d9 lj_vm_dyn.o
lj_buf_tostr 0x3545 lj_vm_dyn.o
lj_buf_putstr_reverse 0x3465 lj_vm_dyn.o
lj_str_new 0x3335 lj_vm_dyn.o
lj_buf_putstr_upper 0x353d lj_vm_dyn.o
lj_dispatch_call 0x3aec lj_vm_dyn.o
lj_trace_hot 0x3abd lj_vm_dyn.o
lj_dispatch_ins 0x3a58 lj_vm_dyn.o
lj_gc_step 0x39e5 lj_vm_dyn.o
lj_err_trace 0x3d99 lj_vm_dyn.o
lj_trace_exit 0x3cbb lj_vm_dyn.o
lj_dispatch_profile 0x3bd0 lj_vm_dyn.o
lj_dispatch_stitch 0x3ba6 lj_vm_dyn.o
lj_err_unwind_dwarf 0x12 lj_vm_dyn.o
lj_ccallback_leave 0x4080 lj_vm_dyn.o
lj_ccallback_enter 0x401f lj_vm_dyn.o
ld: fatal: relocations remain against allocatable but non-writable sections
collect2: error: ld returned 1 exit status
gmake[1]: *** [Makefile:737: libluajit.so] Error 1
gmake[1]: Leaving directory '/ws/safari/luajit/src'
gmake: *** [Makefile:126: default] Error 2These relocations are ostensibly all of type R_AMD64_PC32:
$ elfdump -r src/lj_vm_dyn.o
Relocation Section: .rela.text
type offset addend section symbol
R_AMD64_PC32 0x63c 0xfffffffffffffffc .rela.text lj_tab_len
R_AMD64_PLT32 0xa2e 0xfffffffffffffffc .rela.text pow
R_AMD64_PC32 0xa71 0xfffffffffffffffc .rela.text lj_meta_cat
R_AMD64_PC32 0xc20 0xfffffffffffffffc .rela.text lj_gc_barrieruv
R_AMD64_PC32 0xc88 0xfffffffffffffffc .rela.text lj_gc_barrieruv
R_AMD64_PC32 0xd15 0xfffffffffffffffc .rela.text lj_func_closeuv
R_AMD64_PC32 0xd55 0xfffffffffffffffc .rela.text lj_func_newL_gc
R_AMD64_PC32 0xdba 0xfffffffffffffffc .rela.text lj_tab_new
R_AMD64_PC32 0xdf5 0xfffffffffffffffc .rela.text lj_gc_step_fixtop
R_AMD64_PC32 0xe28 0xfffffffffffffffc .rela.text lj_tab_dup
R_AMD64_PC32 0xe5c 0xfffffffffffffffc .rela.text lj_gc_step_fixtop
R_AMD64_PC32 0x120d 0xfffffffffffffffc .rela.text lj_tab_newkey
R_AMD64_PC32 0x133a 0xfffffffffffffffc .rela.text lj_tab_reasize
R_AMD64_PC32 0x16c8 0xfffffffffffffffc .rela.text lj_state_growstack
R_AMD64_PC32 0x1e31 0xfffffffffffffffc .rela.text lj_state_growstack
R_AMD64_PC32 0x1eed 0xfffffffffffffffc .rela.text lj_state_growstack
R_AMD64_PC32 0x21b8 0xfffffffffffffffc .rela.text lj_meta_tget
R_AMD64_PC32 0x2211 0xfffffffffffffffc .rela.text lj_tab_getinth
R_AMD64_PC32 0x22a1 0xfffffffffffffffc .rela.text lj_meta_tset
R_AMD64_PC32 0x2314 0xfffffffffffffffc .rela.text lj_tab_setinth
R_AMD64_PC32 0x234a 0xfffffffffffffffc .rela.text lj_meta_comp
R_AMD64_PC32 0x23c7 0xfffffffffffffffc .rela.text lj_meta_equal
R_AMD64_PC32 0x23e6 0xfffffffffffffffc .rela.text lj_meta_equal_cd
R_AMD64_PC32 0x2405 0xfffffffffffffffc .rela.text lj_meta_istype
R_AMD64_PC32 0x245a 0xfffffffffffffffc .rela.text lj_meta_arith
R_AMD64_PC32 0x249d 0xfffffffffffffffc .rela.text lj_meta_len
R_AMD64_PC32 0x24ca 0xfffffffffffffffc .rela.text lj_meta_call
R_AMD64_PC32 0x2525 0xfffffffffffffffc .rela.text lj_meta_for
R_AMD64_PC32 0x273a 0xfffffffffffffffc .rela.text lj_tab_get
R_AMD64_PC32 0x27e3 0xfffffffffffffffc .rela.text lj_strfmt_num
R_AMD64_PC32 0x2831 0xfffffffffffffffc .rela.text lj_tab_next
R_AMD64_PC32 0x294e 0xfffffffffffffffc .rela.text lj_tab_getinth
R_AMD64_PC32 0x2bd4 0xfffffffffffffffc .rela.text lj_state_growstack
R_AMD64_PC32 0x2d04 0xfffffffffffffffc .rela.text lj_ffh_coroutine_wrap_err
R_AMD64_PC32 0x2d17 0xfffffffffffffffc .rela.text lj_state_growstack
R_AMD64_PLT32 0x2e55 0xfffffffffffffffc .rela.text log
R_AMD64_PLT32 0x2e83 0xfffffffffffffffc .rela.text log10
R_AMD64_PLT32 0x2eb1 0xfffffffffffffffc .rela.text exp
R_AMD64_PLT32 0x2edf 0xfffffffffffffffc .rela.text sin
R_AMD64_PLT32 0x2f0d 0xfffffffffffffffc .rela.text cos
R_AMD64_PLT32 0x2f3b 0xfffffffffffffffc .rela.text tan
R_AMD64_PLT32 0x2f69 0xfffffffffffffffc .rela.text asin
R_AMD64_PLT32 0x2f97 0xfffffffffffffffc .rela.text acos
R_AMD64_PLT32 0x2fc5 0xfffffffffffffffc .rela.text atan
R_AMD64_PLT32 0x2ff3 0xfffffffffffffffc .rela.text sinh
R_AMD64_PLT32 0x3021 0xfffffffffffffffc .rela.text cosh
R_AMD64_PLT32 0x304f 0xfffffffffffffffc .rela.text tanh
R_AMD64_PLT32 0x3094 0xfffffffffffffffc .rela.text pow
R_AMD64_PLT32 0x30d9 0xfffffffffffffffc .rela.text atan2
R_AMD64_PLT32 0x311e 0xfffffffffffffffc .rela.text fmod
R_AMD64_PLT32 0x3191 0xfffffffffffffffc .rela.text frexp
R_AMD64_PLT32 0x31dd 0xfffffffffffffffc .rela.text modf
R_AMD64_PC32 0x3335 0xfffffffffffffffc .rela.text lj_str_new
R_AMD64_PC32 0x3465 0xfffffffffffffffc .rela.text lj_buf_putstr_reverse
R_AMD64_PC32 0x346d 0xfffffffffffffffc .rela.text lj_buf_tostr
R_AMD64_PC32 0x34d1 0xfffffffffffffffc .rela.text lj_buf_putstr_lower
R_AMD64_PC32 0x34d9 0xfffffffffffffffc .rela.text lj_buf_tostr
R_AMD64_PC32 0x353d 0xfffffffffffffffc .rela.text lj_buf_putstr_upper
R_AMD64_PC32 0x3545 0xfffffffffffffffc .rela.text lj_buf_tostr
R_AMD64_PC32 0x39b9 0xfffffffffffffffc .rela.text lj_state_growstack
R_AMD64_PC32 0x39e5 0xfffffffffffffffc .rela.text lj_gc_step
R_AMD64_PC32 0x3a58 0xfffffffffffffffc .rela.text lj_dispatch_ins
R_AMD64_PC32 0x3abd 0xfffffffffffffffc .rela.text lj_trace_hot
R_AMD64_PC32 0x3aec 0xfffffffffffffffc .rela.text lj_dispatch_call
R_AMD64_PC32 0x3ba6 0xfffffffffffffffc .rela.text lj_dispatch_stitch
R_AMD64_PC32 0x3bd0 0xfffffffffffffffc .rela.text lj_dispatch_profile
R_AMD64_PC32 0x3cbb 0xfffffffffffffffc .rela.text lj_trace_exit
R_AMD64_PC32 0x3d99 0xfffffffffffffffc .rela.text lj_err_trace
R_AMD64_PC32 0x401f 0xfffffffffffffffc .rela.text lj_ccallback_enter
R_AMD64_PC32 0x4080 0xfffffffffffffffc .rela.text lj_ccallback_leave
R_AMD64_PC32 0x1e6b 0xfffffffffffffffc .rela.text lj_err_throw
Relocation Section: .rela.debug_frame
type offset addend section symbol
R_AMD64_32 0x1c 0 .rela.debug_fr .debug_frame (section)
R_AMD64_64 0x20 0 .rela.debug_fr .text (section)
R_AMD64_32 0x44 0 .rela.debug_fr .debug_frame (section)
R_AMD64_64 0x48 0 .rela.debug_fr lj_vm_ffi_call
Relocation Section: .rela.eh_frame
type offset addend section symbol
R_AMD64_PC32 0x12 0 .rela.eh_frame lj_err_unwind_dwarf
R_AMD64_PC32 0x28 0 .rela.eh_frame .text (section)
R_AMD64_PC32 0x60 0 .rela.eh_frame lj_vm_ffi_callOf note, src/lj_vm_dyn.o appears to be built from a very special assembler, hand-crafted for this project! e.g., the routine lj_vmeta_equal():
lj_meta_equal 0x23c7 lj_vm_dyn.o
....
lj_vmeta_equal()
0x23a1: 48 c1 e0 11 shlq $0x11,%rax
0x23a5: 48 c1 e8 11 shrq $0x11,%rax
0x23a9: 48 83 eb 04 subq $0x4,%rbx
0x23ad: 48 89 ce movq %rcx,%rsi
0x23b0: 89 e9 movl %ebp,%ecx
0x23b2: 48 8b 6c 24 10 movq 0x10(%rsp),%rbp
0x23b7: 48 89 55 20 movq %rdx,0x20(%rbp)
0x23bb: 48 89 c2 movq %rax,%rdx
0x23be: 48 89 ef movq %rbp,%rdi
0x23c1: 48 89 5c 24 18 movq %rbx,0x18(%rsp)
0x23c6: e8 00 00 00 00 call +0x0 <0x23cb> <----- relocation here
0x23cb: eb 81 jmp -0x7f <0x234e>comes from src/vm_x64.dasc:
|->vmeta_equal:
| cleartp TAB:RD
| sub PC, 4
|.if X64WIN
| mov CARG3, RD
| mov CARG4d, RBd
| mov L:RB, SAVE_L
| mov L:RB->base, BASE // Caveat: CARG2 == BASE.
| mov CARG2, RA
| mov CARG1, L:RB // Caveat: CARG1 == RA.
|.else
| mov CARG2, RA
| mov CARG4d, RBd // Caveat: CARG4 == RA.
| mov L:RB, SAVE_L
| mov L:RB->base, BASE // Caveat: CARG3 == BASE.
| mov CARG3, RD
| mov CARG1, L:RB
|.endif
| mov SAVE_PC, PC
| call extern lj_meta_equal // (lua_State *L, GCobj *o1, *o2, int ne)
| // 0/1 or TValue * (metamethod) returned in eax (RC).
| jmp <3The special assembler is invoked thus:
BUILDVM lj_vm.S
host/buildvm -m elfasm -o lj_vm.S
ASM lj_vm.o
gcc -fPIC -O2 -fomit-frame-pointer -Wall -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -U_FORTIFY_SOURCE -DLUA_MULTILIB=\"lib\" -DLUA_LJDIR=\"/usr/local/share/luajit-2.1\" -fno-stack-protector -DLUAJIT_UNWIND_EXTERNAL -c -o lj_vm_dyn.o lj_vm.S
gcc -O2 -fomit-frame-pointer -Wall -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -U_FORTIFY_SOURCE -DLUA_MULTILIB=\"lib\" -DLUA_LJDIR=\"/usr/local/share/luajit-2.1\" -fno-stack-protector -DLUAJIT_UNWIND_EXTERNAL -c -o lj_vm.o lj_vm.SIt produces assembly files that it then gives to GCC; e.g., for the function shown above:
.globl lj_vmeta_equal
.hidden lj_vmeta_equal
.type lj_vmeta_equal, @function
.size lj_vmeta_equal, 44
lj_vmeta_equal:
.byte 72,193,224,17,72,193,232,17,72,131,235,4,72,137,206,137
.byte 233,72,139,108,36,16,72,137,85,32,72,137,194,72,137,239
.byte 72,137,92,36,24
call lj_meta_equal
.byte 235,129It's conceivable that the generated assembly is not quite right. More study required to determine where the bug is, exactly.
@pfmooney and I have seen random crashes from Neovim 0.10 lately, and I have finally had some cycles to dig in and debug it. The crux of the issue is that the embedded LuaJIT library is doing some relatively daft things with mappings, and depending on how unlucky you get with random numbers, it will collide with the process heap and malloc() will stop being able to allocate more memory.
When this happens, the process exits unceremoniously with somewhat corrupted output:
I used DTrace to get a sense of what was causing the allocation failures. They're all failures from malloc(), and if you look at a regular nvim process that has not yet failed, it's not hard to see why:
Critically, in this particular instance, the heap currently extends up to
0x1205BF0, and theMAP_ANONmappings that LuaJIT has created are as low as0xB3C0000, which is only 161MB away! The initial address that's chosen for the mapping is made up by a random number generator in LuaJIT, and constrained to be within about 2GB of some of the language runtime functions; these are compiled into the binary, so they're down under the heap in the address space. Further compounding the issue, LuaJIT appears to really want to grow its allocation downward, setting it on a collision course with the heap.This doesn't happen as much on, say, Ubuntu, because they are able to build and ship LuaJIT as a shared library. This library ends up loaded way up high in the address space, very far from the heap, and so mappings in that 2GB neighbourhood are less of an issue. We should try to do the same thing, but I believe some investigation may be required to make that work correctly.
In the meantime, it seems possible to work around the issue by preloading libumem and setting UMEM_OPTIONS to backend=mmap in the environment. This causes malloc() to be provided by umem, and to avoid the classic sbrk()-based heap entirely. We can provide a default value for UMEM_OPTIONS by adding a C function with the appropriate name to the resultant binary; e.g.,
Note that this is not a fix, and we should still also investigate building LuaJIT as a shared library, to at least be on the same footing as other common platforms that ship Neovim and LuaJIT.