ncm2 jedi 'too many open files' exception

NCM2 Jedi raises an exception that it cannot open any more files (open file limit set to 128). I was only editing a single file when this problem occurred, which included some standard libraries and numpy.

I was wondering if ncm2 closes files that are not used after a period of time.

OS

mac os high sierra

neovim `:version` output

NVIM v0.3.0 Build type: Release LuaJIT 2.0.5 Compilation: /usr/local/Homebrew/Library/Homebrew/shims/mac/super/clang -Wconversion -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1 -DNVIM_MSGPACK_HAS_FLOAT32 -DNVIM_UNIBI_HAS_VAR_FROM -DNDEBUG -DMIN_LOG_LEVEL=3 -Wall -Wextra -pedantic -Wno-unused-parameter -Wstrict-prototypes -std=gnu99 -Wimplicit-fallthrough -Wvla -fstack-protector -strong -fdiagnostics-color=auto -DINCLUDE_GENERATED_DECLARATIONS -I/tmp/neovim-20180612-71518-1f95nfa/neovim-0.3.0/build/config -I/tmp/neovim-20180612-71518-1f95nfa/neovim-0.3.0/src -I/usr/local/include -I/usr/local/opt/gettext/include -I/usr/include -I/tmp/neovim-20180612-71518-1f95nfa/neovim-0.3.0/build/src/nvim/auto -I/t mp/neovim-20180612-71518-1f95nfa/neovim-0.3.0/build/include Compiled by brew@HighSierra.local

Minimal vimrc for reproducing the issue

call plug#begin('~/.local/share/nvim/plugged')

Plug 'ncm2/ncm2-ultisnips'
Plug 'ncm2/ncm2'

" ncm2 sources (https://github.com/ncm2/ncm2/wiki)
Plug 'ncm2/ncm2-bufword'
Plug 'ncm2/ncm2-tmux'
Plug 'ncm2/ncm2-path'
Plug 'ncm2/ncm2-jedi'
Plug 'ncm2/ncm2-pyclang'

" libs
Plug 'roxma/nvim-yarp'

call plug#end()

" ======== Plugin Config =======

" ncm2
" enable ncm2 for all buffers
autocmd BufEnter * call ncm2#enable_for_buffer()
set completeopt=noinsert,menuone,noselect

" suppress the annoying 'match x of y', 'The only match' and 'Pattern not
" found' messages
set shortmess+=c

" CTRL-C doesn't trigger the InsertLeave autocmd . map to <ESC> instead.
inoremap <c-c> <ESC>

" When the <Enter> key is pressed while the popup menu is visible, it only
" hides the menu. Use this mapping to close the menu and also start a new
" line.
inoremap <expr> <CR> (pumvisible() ? "\<c-y>\<cr>" : "\<CR>")

" Use <TAB> to select the popup menu:
inoremap <expr> <Tab> pumvisible() ? "\<C-n>" : "\<Tab>"
inoremap <expr> <S-Tab> pumvisible() ? "\<C-p>" : "\<S-Tab>"

" wrap existing omnifunc
" Note that omnifunc does not run in background and may probably block the
" editor. If you don't want to be blocked by omnifunc too often, you could
" add 180ms delay before the omni wrapper:
"  'on_complete': ['ncm2#on_complete#delay', 180,
"               \ 'ncm2#on_complete#omni', 'csscomplete#CompleteCSS'],
au User Ncm2Plugin call ncm2#register_source({
        \ 'name' : 'css',
        \ 'priority': 9,
        \ 'subscope_enable': 1,
        \ 'scope': ['css','scss'],
        \ 'mark': 'css',
        \ 'word_pattern': '[\w\-]+',
        \ 'complete_pattern': ':\s*',
        \ 'on_complete': ['ncm2#on_complete#omni', 'csscomplete#CompleteCSS'],
        \ })

" latex with vimtex
au User Ncm2Plugin call ncm2#register_source({
            \ 'name' : 'vimtex',
            \ 'priority': 1,
            \ 'subscope_enable': 1,
            \ 'complete_length': 1,
            \ 'scope': ['tex'],
            \ 'matcher': {'name': 'combine',
            \           'matchers': [
            \               {'name': 'abbrfuzzy', 'key': 'menu'},
            \               {'name': 'prefix', 'key': 'word'},
            \           ]},
            \ 'mark': 'tex',
            \ 'word_pattern': '\w+',
            \ 'complete_pattern': g:vimtex#re#ncm,
            \ 'on_complete': ['ncm2#on_complete#omni', 'vimtex#complete#omnifunc'],
            \ })

neovim `:messages` output when the error occurs

[ncm2_jedi@yarp]   File "/Users/user_name/.pyenv/versions/neovim3/lib/python3.6/site-packages/jedi/cache.py", line 143, in wrapper
[ncm2_jedi@yarp]     result = method(self, *args, **kwargs)
[ncm2_jedi@yarp]   File "/Users/user_name/.pyenv/versions/neovim3/lib/python3.6/site-packages/jedi/evaluate/compiled/subprocess/__init__.py", line 177, in _process
[ncm2_jedi@yarp]     bufsize=-1
[ncm2_jedi@yarp]   File "/Users/user_name/.pyenv/versions/neovim3/lib/python3.6/site-packages/jedi/_compatibility.py", line 525, in __init__
[ncm2_jedi@yarp]     super(GeneralizedPopen, self).__init__(*args, **kwargs)
[ncm2_jedi@yarp]   File "/Users/user_name/.pyenv/versions/3.6.2/lib/python3.6/subprocess.py", line 707, in __init__
[ncm2_jedi@yarp]     restore_signals, start_new_session)
[ncm2_jedi@yarp]   File "/Users/user_name/.pyenv/versions/3.6.2/lib/python3.6/subprocess.py", line 1226, in _execute_child
[ncm2_jedi@yarp]     errpipe_read, errpipe_write = os.pipe()
[ncm2_jedi@yarp] OSError: [Errno 24] Too many open files
[ncm2_jedi@yarp] During handling of the above exception, another exception occurred:
[ncm2_jedi@yarp] Traceback (most recent call last):
[ncm2_jedi@yarp]   File "/Users/user_name/.local/share/nvim/plugged/nvim-yarp/pythonx/yarp.py", line 30, in on_notification
[ncm2_jedi@yarp]     getattr(module_obj, method)(*args)
[ncm2_jedi@yarp]   File "/Users/user_name/.local/share/nvim/plugged/ncm2/pythonx/ncm2.py", line 206, in on_complete
[ncm2_jedi@yarp]     on_complete_impl(context, *args)
[ncm2_jedi@yarp]   File "/Users/user_name/.local/share/nvim/plugged/ncm2-jedi/pythonx/ncm2_jedi.py", line 40, in on_complete
[ncm2_jedi@yarp]     script = jedi.Script(src, lnum, len(typed), path)
[ncm2_jedi@yarp]   File "/Users/user_name/.pyenv/versions/neovim3/lib/python3.6/site-packages/jedi/api/__init__.py", line 108, in __init__
[ncm2_jedi@yarp]     project, environment=environment, script_path=self.path
[ncm2_jedi@yarp]   File "/Users/user_name/.pyenv/versions/neovim3/lib/python3.6/site-packages/jedi/evaluate/__init__.py", line 94, in __init__
[ncm2_jedi@yarp]     self.compiled_subprocess = environment.get_evaluator_subprocess(self)
[ncm2_jedi@yarp]   File "/Users/user_name/.pyenv/versions/neovim3/lib/python3.6/site-packages/jedi/api/environment.py", line 113, in get_evaluator_subprocess
[ncm2_jedi@yarp]     return EvaluatorSubprocess(evaluator, self._get_subprocess())
[ncm2_jedi@yarp]   File "/Users/user_name/.pyenv/versions/neovim3/lib/python3.6/site-packages/jedi/api/environment.py", line 79, in _get_subprocess
[ncm2_jedi@yarp]     exc))
[ncm2_jedi@yarp] jedi.api.environment.InvalidPythonEnvironment: Could not get version information for '/Users/user_name/.pyenv/versions/neovim3/bin/python': OSError(24, 'Too many open files')
[ncm2_jedi@yarp] Exception ignored in: <bound method CompiledSubprocess.__del__ of <CompiledSubprocess _executable='/Users/user_name/.pyenv/versions/neovim3/bin/python', _pickle_protocol=2, is_crashed=True, pid=66727>>
[ncm2_jedi@yarp] Traceback (most recent call last):
[ncm2_jedi@yarp]   File "/Users/user_name/.pyenv/versions/neovim3/lib/python3.6/site-packages/jedi/evaluate/compiled/subprocess/__init__.py", line 217, in __del__
[ncm2_jedi@yarp]     self._kill()
[ncm2_jedi@yarp]   File "/Users/user_name/.pyenv/versions/neovim3/lib/python3.6/site-packages/jedi/evaluate/compiled/subprocess/__init__.py", line 207, in _kill
[ncm2_jedi@yarp]     self._process.kill()
[ncm2_jedi@yarp]   File "/Users/user_name/.pyenv/versions/neovim3/lib/python3.6/site-packages/jedi/cache.py", line 143, in wrapper
[ncm2_jedi@yarp]     result = method(self, *args, **kwargs)
[ncm2_jedi@yarp]   File "/Users/user_name/.pyenv/versions/neovim3/lib/python3.6/site-packages/jedi/evaluate/compiled/subprocess/__init__.py", line 177, in _process
[ncm2_jedi@yarp]     bufsize=-1
[ncm2_jedi@yarp]   File "/Users/user_name/.pyenv/versions/neovim3/lib/python3.6/site-packages/jedi/_compatibility.py", line 525, in __init__
[ncm2_jedi@yarp]     super(GeneralizedPopen, self).__init__(*args, **kwargs)
[ncm2_jedi@yarp]   File "/Users/user_name/.pyenv/versions/3.6.2/lib/python3.6/subprocess.py", line 707, in __init__
[ncm2_jedi@yarp]     restore_signals, start_new_session)
[ncm2_jedi@yarp]   File "/Users/user_name/.pyenv/versions/3.6.2/lib/python3.6/subprocess.py", line 1226, in _execute_child
[ncm2_jedi@yarp]     errpipe_read, errpipe_write = os.pipe()
[ncm2_jedi@yarp] OSError: [Errno 24] Too many open files
[ncm2_jedi@yarp] error caught in notification handler 'on_complete [{'filepath': '/Users/user_name/Work/school/4A/cs486/a3/variable_elimination_algorithm.py', 'typed': '    A=false respectively, by a given a order argument. arg', 'scope_match': 'python', 'startccol': 56, 'scope_level': 1, 'word_pattern': '(-?\\d*\\.\\d\\w*)|([
^\\`\\~\\!\\@\\#\\$\\%\\^\\&\\*\\(\\)\\-\\=\\+\\[\\{\\]\\}\\\\\\|\\;\\:\\\'\\"\\,\\.\\<\\>\\/\\?\\s]+)', 'time': 1541814150.982941, 'lnum': 23, 'bufnr': 1, 'changedtick': 17397, 'manual': 0, 'base': 'arg', 'ccol': 59, 'filetype': 'python', 'curpos': [0, 23, 59, 0, 59], 'tick': [[0, 23, 59], 28], 'early_cache': False, 'contex
...

Log files if you think the issue is a bug

Traceback (most recent call last):
  File "/Users/user_name/.pyenv/versions/neovim3/lib/python3.6/site-packages/jedi/api/environment.py", line 74, in _get_subprocess
    info = self._subprocess._send(None, _get_info)
  File "/Users/user_name/.pyenv/versions/neovim3/lib/python3.6/site-packages/jedi/evaluate/compiled/subprocess/__init__.py", line 229, in _send
    pickle_dump(data, self._process.stdin, self._pickle_protocol)
  File "/Users/user_name/.pyenv/versions/neovim3/lib/python3.6/site-packages/jedi/cache.py", line 143, in wrapper
    result = method(self, *args, **kwargs)
  File "/Users/user_name/.pyenv/versions/neovim3/lib/python3.6/site-packages/jedi/evaluate/compiled/subprocess/__init__.py", line 177, in _process
    bufsize=-1
  File "/Users/user_name/.pyenv/versions/neovim3/lib/python3.6/site-packages/jedi/_compatibility.py", line 525, in __init__
    super(GeneralizedPopen, self).__init__(*args, **kwargs)
  File "/Users/user_name/.pyenv/versions/3.6.2/lib/python3.6/subprocess.py", line 665, in __init__
    errread, errwrite) = self._get_handles(stdin, stdout, stderr)
  File "/Users/user_name/.pyenv/versions/3.6.2/lib/python3.6/subprocess.py", line 1181, in _get_handles
    errread, errwrite = os.pipe()
OSError: [Errno 24] Too many open files

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/user_name/.pyenv/versions/neovim3/lib/python3.6/site-packages/neovim/msgpack_rpc/session.py", line 213, in handler
    self._notification_cb(name, args)
  File "/Users/user_name/.pyenv/versions/neovim3/lib/python3.6/site-packages/neovim/api/nvim.py", line 210, in filter_notification_cb
    notification_cb(name, args)
  File "/Users/user_name/.local/share/nvim/plugged/nvim-yarp/pythonx/yarp.py", line 30, in on_notification
    getattr(module_obj, method)(*args)
  File "/Users/user_name/.local/share/nvim/plugged/ncm2/pythonx/ncm2.py", line 206, in on_complete
    on_complete_impl(context, *args)
  File "/Users/user_name/.local/share/nvim/plugged/ncm2-jedi/pythonx/ncm2_jedi.py", line 40, in on_complete
    script = jedi.Script(src, lnum, len(typed), path)
  File "/Users/user_name/.pyenv/versions/neovim3/lib/python3.6/site-packages/jedi/api/__init__.py", line 108, in __init__
    project, environment=environment, script_path=self.path
  File "/Users/user_name/.pyenv/versions/neovim3/lib/python3.6/site-packages/jedi/evaluate/__init__.py", line 94, in __init__
    self.compiled_subprocess = environment.get_evaluator_subprocess(self)
  File "/Users/user_name/.pyenv/versions/neovim3/lib/python3.6/site-packages/jedi/api/environment.py", line 113, in get_evaluator_subprocess
    return EvaluatorSubprocess(evaluator, self._get_subprocess())
  File "/Users/user_name/.pyenv/versions/neovim3/lib/python3.6/site-packages/jedi/api/environment.py", line 79, in _get_subprocess
    exc))
jedi.api.environment.InvalidPythonEnvironment: Could not get version information for '/Users/user_name/.pyenv/versions/neovim3/bin/python': OSError(24, 'Too many open files')

ncm2 / ncm2