[python] is it possible to run YCM against multiple different python versions?

[viggyfresh]

Issue Prelude

Please complete these steps and check these boxes (by putting an x inside the brackets) before filing your issue:

• [x] I have read and understood YCM's [CONTRIBUTING][cont] document.
• [x] I have read and understood YCM's [CODE_OF_CONDUCT][code] document.
• [x] I have read and understood YCM's [README][readme], especially the [Frequently Asked Questions][faq] section.
• [x] I have searched YCM's issue tracker to find issues similar to the one I'm about to report and couldn't find an answer to my problem. ([Example Google search.][search])
• [x] If filing a bug report, I have included the output of vim --version.
• [x] If filing a bug report, I have included the output of :YcmDebugInfo.
• [x] If filing a bug report, I have attached the contents of the logfiles using the :YcmToggleLogs command.
• [x] If filing a bug report, I have included which OS (including specific OS version) I am using.
• [x] If filing a bug report, I have included a minimal test case that reproduces my issue, using vim -Nu /path/to/YCM/vimrc_ycm_minimal, including what I expected to happen and what actually happened.
• [x] If filing a installation failure report, I have included the entire output of install.py (or cmake/make/ninja) including its invocation
• [x] I understand this is an open-source project staffed by volunteers and that any help I receive is a selfless, heartfelt gift of their free time. I know I am not entitled to anything and will be polite and courteous.
• [x] I understand my issue may be closed if it becomes obvious I didn't actually perform all of these steps.

Thank you for adhering to this process! It ensures your issue is resolved quickly and that neither your nor our time is needlessly wasted.

Issue Details

To whom it may concern - this is a question that I believe I know the answer to - but wanted to make sure that my understanding is correct! I couldn't find a Git issue or a StackOverflow link that came out and explicitly said the behavior I'm hunting for is possible or impossible.

My use case: we swap between multiple major Python versions (e.g. 3.6, 3.7, 3.8) for various microservices we run at my workplace. To make this possible, we use pyenv and pyenv-virtualenv. I'm trying to configure a Docker development image with Ubuntu 20.04 and Vim + python3 + YCM - but when I point YCM to a pyenv Python interpreter (e.g. let g:ycm_path_to_python_interpreter = '/root/.pyenv/shims/python'), I get the following error:

ImportError: Python version mismatch: module was compiled for Python 3.8, but the interpreter version is incompatible: 3.6.10 (default, May 23 2020, 16:57:59)

• What did you do?

1. Install vim and pyenv and YouCompleteMe
2. Add let g:ycm_path_to_python_interpreter = '/root/.pyenv/shims/python' to .vimrc
3. Start VIM, see error

.vimrc:

let g:ycm_path_to_python_interpreter = '/root/.pyenv/shims/python'

• What did you expect to happen?

Not sure if this is supported 😬- but in an ideal world, I'd be able to utilize pyenv and YouCompleteMe together! Is this at all possible?

• What actually happened?

YCM fails to start with error detailed above (and below)

Diagnostic data

Output of vim --version

VIM - Vi IMproved 8.1 (2018 May 18, compiled Apr 15 2020 06:40:31)
Included patches: 1-2269
Modified by team+vim@tracker.debian.org
Compiled by team+vim@tracker.debian.org
Huge version without GUI.  Features included (+) or not (-):
+acl               -farsi             -mouse_sysmouse    -tag_any_white
+arabic            +file_in_path      +mouse_urxvt       -tcl
+autocmd           +find_in_path      +mouse_xterm       +termguicolors
+autochdir         +float             +multi_byte        +terminal
-autoservername    +folding           +multi_lang        +terminfo
-balloon_eval      -footer            -mzscheme          +termresponse
+balloon_eval_term +fork()            +netbeans_intg     +textobjects
-browse            +gettext           +num64             +textprop
++builtin_terms    -hangul_input      +packages          +timers
+byte_offset       +iconv             +path_extra        +title
+channel           +insert_expand     -perl              -toolbar
+cindent           +job               +persistent_undo   +user_commands
-clientserver      +jumplist          +postscript        +vartabs
-clipboard         +keymap            +printer           +vertsplit
+cmdline_compl     +lambda            +profile           +virtualedit
+cmdline_hist      +langmap           -python            +visual
+cmdline_info      +libcall           +python3           +visualextra
+comments          +linebreak         +quickfix          +viminfo
+conceal           +lispindent        +reltime           +vreplace
+cryptv            +listcmds          +rightleft         +wildignore
+cscope            +localmap          -ruby              +wildmenu
+cursorbind        -lua               +scrollbind        +windows
+cursorshape       +menu              +signs             +writebackup
+dialog_con        +mksession         +smartindent       -X11
+diff              +modify_fname      +sound             -xfontset
+digraphs          +mouse             +spell             -xim
-dnd               -mouseshape        +startuptime       -xpm
-ebcdic            +mouse_dec         +statusline        -xsmp
+emacs_tags        +mouse_gpm         -sun_workshop      -xterm_clipboard
+eval              -mouse_jsbterm     +syntax            -xterm_save
+ex_extra          +mouse_netterm     +tag_binary        
+extra_search      +mouse_sgr         -tag_old_static    
   system vimrc file: "$VIM/vimrc"
     user vimrc file: "$HOME/.vimrc"
 2nd user vimrc file: "~/.vim/vimrc"
      user exrc file: "$HOME/.exrc"
       defaults file: "$VIMRUNTIME/defaults.vim"
  fall-back for $VIM: "/usr/share/vim"
Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H   -Wdate-time  -g -O2 -fdebug-prefix-map=/build/vim-iU6mZD/vim-8.1.2269=. -fstack-protector-strong -Wformat -Werror=format-security -D_REENTRANT -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1       
Linking: gcc   -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -o vim        -lm -ltinfo -lnsl  -lselinux  -lcanberra -lacl -lattr -lgpm -ldl     -L/usr/lib/python3.8/config-3.8-x86_64-linux-gnu -lpython3.8 -lcrypt -lpthread -ldl -lutil -lm -lm    

Output of YcmDebugInfo

Printing YouCompleteMe debug information...
-- Completion API: 1
-- Client logfile: /tmp/ycm_0rz49gqm.log
-- Server errored, no debug info from server
-- Server running at: http://127.0.0.1:43947
-- Server process ID: 19992

Output of YcmDiags

N/A

Output of git rev-parse HEAD in YouCompleteMe installation directory

170bae7975de1ca3e291b535a5230dcdb46ae274

Contents of YCM, ycmd and completion engine logfiles

File "/root/.vim/pack/plugins/start/YouCompleteMe/third_party/ycmd/ycmd/utils.py", line 498, in ImportAndCheckCore
ycm_core = ImportCore(
File "/root/.vim/pack/plugins/start/YouCompleteMe/third_party/ycmd/ycmd/utils.py", line 489, in ImportCore
import ycm_core as ycm_core
ImportError: Python version mismatch: module was compiled for Python 3.8, but the interpreter version is incompatible: 3.6.10 (default, May 23 2020, 16:57:59)
[GCC 9.3.0].

OS version, distribution, etc.

Ubuntu 20.04, inside Docker

[Shougo]

Unfortunately, you cannot change the Python interpreter version. It is Vim8's limitation.

If you really want to change the version, neovim is recommended. But YCM is slow in neovim unfortunately due to RPC overhead.

[bstaletic]

There are 3 interpreters at play here.

1. The one running inside vim. This one is impossible to change without recompiling vim itself.
2. The one you used to compile the C++ part of the plugin. If you want to change that one, you'll have to recompile YCM again.
3. Finally, the interpreter used by jedi, for completions and whatnot. This one can easily be changed. Read: https://github.com/ycm-core/YouCompleteMe#working-with-virtual-environments

[viggyfresh]

rats, I see - that’s what I figured. 2 I think is the case that’s biting me, since I’m switching python major versions via pyenv. Will investigate some things and report back

[puremourning]

ycmd remembers the (absolute) path to the python interpreter used to build it. So if you use an abs path to one that won't move when you built it is should be fine. So long as it's a supported version, it won't matter which one you use, so best use your system python, e.b. /usr/bin/python3 install.py ....

If you really need to use a pyenv to build ycmd, then it's a bit tricker, but you might be able to specifically reference the $HOME/.pyenv/versions/.../bin/python3 install.py or something, but using the system one should probably work.

[viggyfresh]

thanks so much everyone for your timely responses! this is a bad misunderstanding on my part.

I was adding the following snippet to my .vimrc:

let g:ycm_path_to_python_interpreter = '/root/.pyenv/shims/python'

however, what I needed to set was the following:

let g:ycm_python_binary_path = '/root/.pyenv/shims/python'

The former only worked when the pyenv shim's python major version matched what Vim was compiled with (3.8). The latter appears to work even when Vim is compiled with 3.8 and my virtualenv is at 3.6!

Does this behavior make sense? if so, this issue can be happily closed :)

[bstaletic]

Yes, that makes sense, however the g:ycm_python_binary_path is not documented for a reason. It's deprecated and you're supposed to configure this in your .ycm_extra_conf.py. If your project needs a customized sys.path, .ycm_extra_conf.py can do that as well.

def Settings( **kwargs ):
  return {
    'sys_path': [
      '/path/to/some/third_party/package',
      '/path/to/another/third_party/package'
    ],
    'interpreter_path': '/path/to/virtual/environment/python'
  }

[viggyfresh]

Got it, thanks! Will do.