completor throws exception while completing c++ sources

[dahakawang]

The bug is easy to trigger, just open an c++ source file, press and hold 'd'.

I met this issue after done an upgrade( now I'm at 4ef67a0 but not sure where I was), I'm using osx and my VIM info is also attached.

appreciate if you can help to solve this bug, and let me know if you need further input from me. 😃

VIM INFO

VIM - Vi IMproved 8.1 (2018 May 18, compiled May  8 2019 21:19:47)
macOS version
Included patches: 1-1300
Compiled by Homebrew
Huge version without GUI.  Features included (+) or not (-):
+acl               +extra_search      +mouse_netterm     -tag_old_static
+arabic            -farsi             +mouse_sgr         -tag_any_white
+autocmd           +file_in_path      -mouse_sysmouse    -tcl
+autochdir         +find_in_path      +mouse_urxvt       +termguicolors
-autoservername    +float             +mouse_xterm       +terminal
-balloon_eval      +folding           +multi_byte        +terminfo
+balloon_eval_term -footer            +multi_lang        +termresponse
-browse            +fork()            -mzscheme          +textobjects
++builtin_terms    +gettext           +netbeans_intg     +textprop
+byte_offset       -hangul_input      +num64             +timers
+channel           +iconv             +packages          +title
+cindent           +insert_expand     +path_extra        -toolbar
-clientserver      +job               +perl              +user_commands
+clipboard         +jumplist          +persistent_undo   +vartabs
+cmdline_compl     +keymap            +postscript        +vertsplit
+cmdline_hist      +lambda            +printer           +virtualedit
+cmdline_info      +langmap           +profile           +visual
+comments          +libcall           -python            +visualextra
+conceal           +linebreak         +python3           +viminfo
+cryptv            +lispindent        +quickfix          +vreplace
+cscope            +listcmds          +reltime           +wildignore
+cursorbind        +localmap          +rightleft         +wildmenu
+cursorshape       +lua               +ruby              +windows
+dialog_con        +menu              +scrollbind        +writebackup
+diff              +mksession         +signs             -X11
+digraphs          +modify_fname      +smartindent       -xfontset
-dnd               +mouse             +startuptime       -xim
-ebcdic            -mouseshape        +statusline        -xpm
+emacs_tags        +mouse_dec         -sun_workshop      -xsmp
+eval              -mouse_gpm         +syntax            -xterm_clipboard
+ex_extra          -mouse_jsbterm     +tag_binary        -xterm_save
   system vimrc file: "$VIM/vimrc"
     user vimrc file: "$HOME/.vimrc"
 2nd user vimrc file: "~/.vim/vimrc"
      user exrc file: "$HOME/.exrc"
       defaults file: "$VIMRUNTIME/defaults.vim"
  fall-back for $VIM: "/usr/local/share/vim"
Compilation: clang -c -I. -Iproto -DHAVE_CONFIG_H   -DMACOS_X -DMACOS_X_DARWIN  -g -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1
Linking: clang   -L. -fstack-protector-strong -L/usr/local/lib -L/usr/local/opt/libyaml/lib -L/usr/local/opt/openssl/lib -L/usr/local/opt/readline/lib  -L/usr/local/lib -o vim        -lncurses -liconv -lintl -framework AppKit  -L/usr/local/opt/lua/lib -llua5.3 -mmacosx-version-min=10.14 -fstack-protector-strong -L/usr/local/lib  -L/usr/local/Cellar/perl/5.28.1/lib/perl5/5.28.1/darwin-thread-multi-2level/CORE -lperl -lm -lutil -lc  -L/usr/local/opt/python/Frameworks/Python.framework/Versions/3.7/lib/python3.7/config-3.7m-darwin -lpython3.7m -framework CoreFoundation  -lruby.2.6

Error Message

unknown type name 'dddddddddddddddd'
unknown type name 'dddddddddddddddd'
--No lines in buffer--
unknown type name 'dddddddddddddddd'
"a.cc" [New] 4L, 56C written
Error detected while processing function <SNR>136_vim_oneshot_handler[6]..completor#action#callback[1]..completor#utils#on_data:
line    1:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/david/.vim/plugged/completor.vim/pythonx/completor/api.py", line 14, in wrapper
    return func(vim.bindeval('a:'))
  File "/Users/david/.vim/plugged/completor.vim/pythonx/completor/api.py", line 44, in on_data
    return c.on_data(args['action'], args['msg']) if c else []
  File "/Users/david/.vim/plugged/completor.vim/pythonx/completor/__init__.py", line 239, in on_data
    return self.do_complete(data)
  File "/Users/david/.vim/plugged/completor.vim/pythonx/completor/__init__.py", line 199, in do_complete
    ret.extend(self.on_complete(data))
  File "/Users/david/.vim/plugged/completor.vim/pythonx/completers/cpp/__init__.py", line 230, in on_complete
    data['word'] = get_word(parts[2])
  File "/Users/david/.vim/plugged/completor.vim/pythonx/completers/cpp/__init__.py", line 52, in get_word
    parts = re.split(r'[ (\[{<]', text, 1)
  File "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.7/lib/python3.7/re.py", line 213, in split
    return _compile(pattern, flags).split(string, maxsplit)
TypeError: cannot use a string pattern on a bytes-like object
Error detected while processing function <SNR>136_vim_oneshot_handler[6]..completor#action#callback[2]..completor#action#trigger[6]..<SNR>131_trigger_complete:
line    7:
E474: Invalid argument
Error detected while processing function <SNR>136_vim_oneshot_handler[6]..completor#action#callback[2]..completor#action#trigger:
line    6:
E171: Missing :endif
unused label 'dddd' [-Wunused-label]

[dahakawang]

similar issue could also be triggered (but not always) if we input main.<TAB>, provide main is a function name defined in current file.

[melko]

I get the same error when I start to type #include, bisecting seems the commit that introduced the issue is 2551f95372f66157cac3e7697d30a351e80ced80

[melko]

As a quick fix I've decoded to utf-8 the byte-string object where needed:

diff --git a/pythonx/completers/cpp/__init__.py b/pythonx/completers/cpp/__init__.py
index a65461511903..83dda755384b 100644
--- a/pythonx/completers/cpp/__init__.py
+++ b/pythonx/completers/cpp/__init__.py
@@ -49,10 +49,10 @@ def strip_optional(menu):

 def get_word(text):
-    parts = re.split(r'[ (\[{<]', text, 1)
+    parts = re.split(r'[ (\[{<]', text.decode('utf-8'), 1)
     if not parts:
         return text
-    return parts[0]
+    return parts[0].encode('utf-8')

 def get_token_path(line, column, word):

or even better I guess:

diff --git a/pythonx/completers/cpp/__init__.py b/pythonx/completers/cpp/__init__.py
index a65461511903..fde637e3a9ac 100644
--- a/pythonx/completers/cpp/__init__.py
+++ b/pythonx/completers/cpp/__init__.py
@@ -49,7 +49,7 @@ def strip_optional(menu):

 def get_word(text):
-    parts = re.split(r'[ (\[{<]', text, 1)
+    parts = re.split(br'[ (\[{<]', text, 1)
     if not parts:
         return text
     return parts[0]

[melko]

As far as I can see this is now fixed in ecb1bf10e55357c9f3ffdce3fe01950747e9aed4 so this issue can be closed.