Segmentation fault with if_lua (during or after the GC)

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Install a plugin which makes use of if_lua (e.g. neocomplete)
2. Wait some hours or days, until the lua GC kicks in
3. Watch it crash

Is there a way to force the GC in vim? I tried to evaluate "collectgarbage()"
and some tricks from here [1], but that didn't reproduce the crash.

[1] http://luatut.com/collectgarbage.html

What is the expected output? No Segfault
What do you see instead? A segfault

What version of the product are you using? On what operating system?

VIM - Vi IMproved 7.4 (2013 Aug 10, compiled Sep 24 2014 05:04:14)
Included patches: 1-459
Compiled by Arch Linux
Huge version with GTK2 GUI.  Features included (+) or not (-):
+acl             +farsi           +mouse_netterm   +syntax
+arabic          +file_in_path    +mouse_sgr       +tag_binary
+autocmd         +find_in_path    -mouse_sysmouse  +tag_old_static
+balloon_eval    +float           +mouse_urxvt     -tag_any_white
+browse          +folding         +mouse_xterm     -tcl
++builtin_terms  -footer          +multi_byte      +terminfo
+byte_offset     +fork()          +multi_lang      +termresponse
+cindent         +gettext         -mzscheme        +textobjects
+clientserver    -hangul_input    +netbeans_intg   +title
+clipboard       +iconv           +path_extra      +toolbar
+cmdline_compl   +insert_expand   +perl            +user_commands
+cmdline_hist    +jumplist        +persistent_undo +vertsplit
+cmdline_info    +keymap          +postscript      +virtualedit
+comments        +langmap         +printer         +visual
+conceal         +libcall         +profile         +visualextra
+cryptv          +linebreak       +python          +viminfo
+cscope          +lispindent      -python3         +vreplace
+cursorbind      +listcmds        +quickfix        +wildignore
+cursorshape     +localmap        +reltime         +wildmenu
+dialog_con_gui  +lua             +rightleft       +windows
+diff            +menu            +ruby            +writebackup
+digraphs        +mksession       +scrollbind      +X11
+dnd             +modify_fname    +signs           -xfontset
-ebcdic          +mouse           +smartindent     +xim
+emacs_tags      +mouseshape      -sniff           +xsmp_interact
+eval            +mouse_dec       +startuptime     +xterm_clipboard
+ex_extra        +mouse_gpm       +statusline      -xterm_save
+extra_search    -mouse_jsbterm   -sun_workshop    -xpm
   system vimrc file: "/etc/vimrc"
     user vimrc file: "$HOME/.vimrc"
 2nd user vimrc file: "~/.vim/vimrc"
      user exrc file: "$HOME/.exrc"
  system gvimrc file: "/etc/gvimrc"
    user gvimrc file: "$HOME/.gvimrc"
2nd user gvimrc file: "~/.vim/gvimrc"
    system menu file: "$VIMRUNTIME/menu.vim"
  fall-back for $VIM: "/usr/share/vim"
Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H -DFEAT_GUI_GTK  -pthread 
-I/usr/include/gtk-2.0 -I/usr/lib/gtk-2.0/include -I/usr/include/pango-1.0 
-I/usr/include/atk-1.0 -I/usr/include/cairo -I/usr/include/pixman-1 
-I/usr/include/libdrm -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/libpng16 
-I/usr/include/pango-1.0 -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include 
-I/usr/include/freetype2 -I/usr/include/libpng16 -I/usr/include/harfbuzz 
-I/usr/include/freetype2 -I/usr/include/libpng16 -I/usr/include/harfbuzz  
-D_FORTIFY_SOURCE=2  -march=x86-64 -mtune=generic -O2 -pipe 
-fstack-protector-strong --param=ssp-buffer-size=4 -U_FORTIFY_SOURCE 
-D_FORTIFY_SOURCE=1      
Linking: gcc   -L. -Wl,-O1,--sort-common,--as-needed,-z,relro -fstack-protector 
-rdynamic -Wl,-export-dynamic -Wl,-E -Wl,-rpath,/usr/lib/perl5/core_perl/CORE  
-Wl,-O1,--sort-common,--as-needed,-z,relro -L/usr/local/lib -Wl,--as-needed -o 
vim   -lgtk-x11-2.0 -lgdk-x11-2.0 -lpangocairo-1.0 -latk-1.0 -lcairo 
-lgdk_pixbuf-2.0 -lgio-2.0 -lpangoft2-1.0 -lpango-1.0 -lgobject-2.0 -lglib-2.0 
-lfontconfig -lfreetype  -lSM -lICE -lXt -lX11 -lXdmcp -lSM -lICE  -lm 
-lncurses -lelf -lnsl   -lacl -lattr -lgpm -ldl  -L/usr/lib -llua -Wl,-E 
-Wl,-rpath,/usr/lib/perl5/core_perl/CORE 
-Wl,-O1,--sort-common,--as-needed,-z,relro -fstack-protector -L/usr/local/lib  
-L/usr/lib/perl5/core_perl/CORE -lperl -lnsl -ldl -lm -lcrypt -lutil -lpthread 
-lc -L/usr/lib/python2.7/config -lpython2.7 -lpthread -ldl -lutil -lm -Xlinker 
-export-dynamic   -lruby -lpthread -lgmp -ldl -lcrypt -lm  -L/usr/lib   

Please provide any additional information below.

Backtrace with gdb, obtained from a core dump

Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f0630571097 in kill () from /usr/lib/libc.so.6
(gdb) bt
#0  0x00007f0630571097 in kill () from /usr/lib/libc.so.6
#1  0x0000000000529580 in mch_exit ()
#2  <signal handler called>
#3  0x0000000000462838 in set_ref_in_item ()
#4  0x00000000005cbb15 in ?? ()
#5  0x00007f0630ed1d5d in ?? () from /usr/lib/liblua.so.5.2
#6  0x00007f0630ed20bd in ?? () from /usr/lib/liblua.so.5.2
#7  0x00007f0630ece2c8 in lua_callk () from /usr/lib/liblua.so.5.2
#8  0x000000000046bb0d in garbage_collect ()
#9  0x000000000052b130 in mch_inchar ()
#10 0x00000000005a5588 in ui_inchar ()
#11 0x00000000004bde5f in inchar ()
#12 0x00000000004bfe81 in ?? ()
#13 0x00000000004c0646 in vgetc ()
#14 0x00000000004c0a99 in safe_vgetc ()
#15 0x000000000050c5dd in normal_cmd ()
#16 0x00000000005e5abd in main_loop ()
#17 0x000000000043a4c1 in main ()

Original issue reported on code.google.com by Jochen.K...@gmail.com on 14 Oct 2014 at 9:38

[GoogleCodeExporter]

Same here, use neocomplete with if_lua, vim simply crash if I don't use it for 
a while.

Original comment by plus...@gmail.com on 31 Oct 2014 at 7:36

[GoogleCodeExporter]

I can't reproduce it...

Is there any way to reproduce the issue?

Original comment by chrisbr...@googlemail.com on 2 Nov 2014 at 2:10

[GoogleCodeExporter]

It's somewhat random (as it is related to GC). For me, when I have vim with 
if_lua compiled in, and install neocomplete, if I open a vim window (this 
happens for both vim in terminal and macvim), do some editing and then leave it 
there, for some time without touching it, maybe several hours or even a day, 
the next time I switch back to that window, it *might* then crash. After 
removing the neocomplete plugin vim never crashed again.

Original comment by plus...@gmail.com on 2 Nov 2014 at 6:27

[GoogleCodeExporter]

I concur that it's random and hard to reproduce. Neocomplete triggers the crash 
for me as well, but I don't think that neocomplete is to blame here. Is there a 
reliable way to stress test the lua gc by evaluating lua code?

Original comment by Jochen.K...@gmail.com on 2 Nov 2014 at 7:19

[GoogleCodeExporter]

you can do :lua collectgarbage("collect"), but that does not crash vim for me.

Original comment by plus...@gmail.com on 2 Nov 2014 at 9:19

[GoogleCodeExporter]

FWIW here's the issue at neocomplete's github page: 
https://github.com/Shougo/neocomplete.vim/issues/310

Original comment by Jochen.K...@gmail.com on 4 Nov 2014 at 10:28

[GoogleCodeExporter]

Following the steps from the issue, this should reproduce a crash:
:" Create some vim object.
:lua for i = 1,1000 do vim.eval('[]') end

:" Execute Lua's incremental garbage collection partly.
:" (Second argument might need to be tuned to reproduce SEGV)
:lua collectgarbage('step', 50)

:" ... Wait 'updatetime' to execute Vim's garbage collection.

:" Execute Lua's garbage collection until finish.
:lua collectgarbage('collect')

unfortunately, I still do not see that.

Original comment by chrisbr...@googlemail.com on 4 Nov 2014 at 7:25

[GoogleCodeExporter]

I think Shuogo (the author of neocomplete) pasted that snippet as an example 
for "finding ways to reproduce the bug without neocomplete". Reading from a 
related issue (https://github.com/vim-jp/issues/issues/352 , unfortunately, 
it's in Japanese), it seems that snippet was used to trigger a similar segfault 
in if_lua, which should have already been fixed in 2013: 
https://groups.google.com/forum/#!msg/vim_dev/4LcTD_3msak/fCzh004mCIYJ

What is not clear is whether that old issue is related to the current one. 

Original comment by plus...@gmail.com on 4 Nov 2014 at 7:55

[GoogleCodeExporter]

I think I jumped to quickly at Lua. From the BT I think the problem lies either 
within vim's own garbage collector or in the set_ref_in_item() [1] function.

I found a discussion which sounds familiar at [2]. However, I tried out the 
vimscript snippet and it does not cause a segfault. In the meantime, I'm still 
getting regularly segfaults using the neocomplete plugin.

Any hints on how to build vim with debugging symbols?

[1] http://fossies.org/dox/vim-7.4/eval_8c_source.html#l06944
[2] https://groups.google.com/d/msg/vim_dev/Nr8Ja4Zjghw/3FfMjVobHmoJ

Original comment by Jochen.K...@gmail.com on 4 Nov 2014 at 9:57

[GoogleCodeExporter]

Try to reproduce the bug after building vim with ASan (= address sanitizer) 
using the following patch to src/Makefile. Then run vim normally, ASan will 
report useful stacks if a memory bug is detected at runtime (double free, used 
of free memory, heap/stack/global overflow)  Make sure that you run vim 
unstripped which should be the case if you ran src/vim. It assumes that you 
compile with a compiler that is not too ancient (>= gcc-4.8 or or >= clang-3.3 
should be fine):

diff -r bfecd68c04dd src/Makefile
--- a/src/Makefile  Fri Oct 31 19:51:36 2014 +0100
+++ b/src/Makefile  Wed Nov 05 04:15:14 2014 +0100
@@ -616,6 +616,11 @@
 #PROFILE_LIBS = -pg
 #PROFILE_LIBS = -pg -lc

+# Build with address sanitizer.
+# See: # https://code.google.com/p/address-sanitizer/wiki/AddressSanitizer
+ASAN_CFLAGS = -fsanitize=address -fno-omit-frame-pointer -O1 -g
+ASAN_LIBS = $(ASAN_CFLAGS)
+
 # MEMORY LEAK DETECTION
 # Requires installing the ccmalloc library.
 # Configuration is in the .ccmalloc or ~/.ccmalloc file.
@@ -1342,7 +1347,7 @@
 PRE_DEFS = -Iproto $(DEFS) $(GUI_DEFS) $(GUI_IPATH) $(CPPFLAGS) $(EXTRA_IPATHS)
 POST_DEFS = $(X_CFLAGS) $(MZSCHEME_CFLAGS) $(TCL_CFLAGS) $(EXTRA_DEFS)

-ALL_CFLAGS = $(PRE_DEFS) $(CFLAGS) $(PROFILE_CFLAGS) $(LEAK_CFLAGS) 
$(POST_DEFS)
+ALL_CFLAGS = $(PRE_DEFS) $(CFLAGS) $(PROFILE_CFLAGS) $(ASAN_CFLAGS) 
$(LEAK_CFLAGS) $(POST_DEFS)

 # Exclude $CFLAGS for osdef.sh, for Mac 10.4 some flags don't work together
 # with "-E".
@@ -1374,6 +1379,7 @@
       $(TCL_LIBS) \
       $(RUBY_LIBS) \
       $(PROFILE_LIBS) \
+      $(ASAN_LIBS) \
       $(LEAK_LIBS)

 # abbreviations

Original comment by dominiqu...@gmail.com on 5 Nov 2014 at 3:23

[GoogleCodeExporter]

> I think I jumped to quickly at Lua. From the BT I think the problem lies 
either > within vim's own garbage collector or in the set_ref_in_item() [1] 
function.
> 
> I found a discussion which sounds familiar at [2]. However, I tried out the 
vimscript snippet and it does not cause a segfault. In the meantime, I'm still 
getting regularly segfaults using the neocomplete plugin.
> 
> Any hints on how to build vim with debugging symbols?
> 
> [1] http://fossies.org/dox/vim-7.4/eval_8c_source.html#l06944
> [2] https://groups.google.com/d/msg/vim_dev/Nr8Ja4Zjghw/3FfMjVobHmoJ

If it is actually the same crash, I made a patch, but it is not complete yet 
for LUA because I did not know how to get a return value from LUA code in the 
case of a failed garbage collect.

To get the snippet in that thread to cause a crash, you may need to increase 
the number of iterations used in the script.

Here is the discussion on the patch:

  https://groups.google.com/d/topic/vim_dev/dnN58kO5Vg4/discussion

Here is the discussion of remaining work:

  https://groups.google.com/forum/#!topic/vim_dev/RzHGXmVewvc

Original comment by benjamin...@rockwellcollins.com on 5 Nov 2014 at 3:46

[GoogleCodeExporter]

This is nuts. I've recompiled vim with the address sanitizer and now it won't 
crash anymore (despite being dog slow). If I simply switch back to my distro's 
binary it'll crash again.

If I find the time I'll give Ben's patch a try.

Original comment by Jochen.K...@gmail.com on 14 Nov 2014 at 11:54

[GoogleCodeExporter]

> This is nuts. I've recompiled vim with the address sanitizer and now it won't 
crash anymore (despite being dog slow). If I simply switch back to my distro's 
binary it'll crash again.

That's quite normal in situations like this one. That could actually suggest 
that it is a concurrency(thread) issue. I would look at locking of variables, 
etc ...

Original comment by kybul...@gmail.com on 2 Jan 2015 at 1:48

[GoogleCodeExporter]

I sort of doubt it's the same crash, but see if the patch for the other garbage 
collection "set_ref" crash makes any difference:

https://groups.google.com/d/msg/vim_dev/dnN58kO5Vg4/uIfaWyjMdM0J

Original comment by fritzoph...@gmail.com on 22 Jan 2015 at 5:22

[GoogleCodeExporter]

I think this was fixed with 7.4.609. Please confirm.

Original comment by chrisbr...@googlemail.com on 3 Feb 2015 at 5:40

• Changed state: Fixed

[GoogleCodeExporter]

Apologies for not replying earlier. I'm currently running 4.7.617 and did not 
see a crash yet. Before this I compiled vim with the patch from here [1] and it 
also solved the problem for me.

Thanks for your help!

[1] 
http://groups.google.com/forum/#!searchin/vim_dev/GC/vim_dev/DBYOdHQWvqY/1WH04_d
wETIJ

Original comment by Jochen.K...@gmail.com on 22 Feb 2015 at 3:31