jedi-vim does not actually use virtualenv

[skamsie]

Issue

It states in the documentation that it "Supports virtualenv", but It does not work for me.

Steps to reproduce

I have a project directory and inside I have a venv folder that was created with:

virtualenv -p /usr/local/bin/python3 venv

I source this venv with the usual command:

source venv/bin/activate

I install some packages in this venv: hamcrest and requests

I start vim in this directory

result of :! python --version is Python 3.4.1

I go to a file in my project that uses either hamcrest or requests, I do \<leader>d, it works, but when I check the path of the file it went to is actually on/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/hamcrest/__init__.py

I uninstall hamcrest from my system python 2.7.13

I start vim again and go to that file, do \<leader>d:

"jedi-vim: Couldn't find any definitions for this."

My conclusion is that in reality jedi-vim does not actually use the virtual env...

Versions

• jedi-vim: no idea
• Vim version 8
• Python: 2.7, 3.4.1, 3.5.2

[davidhalter]

Jedi's support for virtualenvs is currently very marginal. IMO we still have to wait for davidhalter/jedi#385. Sorry, but I really don't like venv support either. However it will get better and I'm finally starting to have time for it.

[pappasam]

I'm experiencing this identical issue. Hopefully it gets fixed soon.

Assuming no one's actively working on this, and without more information, I assume the problem involves the search path for the virtual environment's site-packages within Jedi; the logic used for jedi.evaluate.sys_path.get_venv_path causes a ton of side effects that are hard for me to trace. Is this part of the code undergoing major refactoring in https://github.com/davidhalter/jedi/issues/385 , or is this something that would be worth my time to wrap my head around and address in Jedi's code base to fix this issue?

Eg, if I dedicate a ton of time for this and come up with a good solution, will the pull request be accepted or is it likely to conflict with the new architectural direction proposed in https://github.com/davidhalter/jedi/issues/385?

[blueyed]

You might want to try https://github.com/davidhalter/jedi/pull/829 in the meantime. Maybe it could be optionally activated, since the "only" reason to not use it seems to be that it can cause crashes?!

[pappasam]

@blueyed wow, thanks a bunch! This completely fixes the problem. I'm curious why this might cause segfaults... @davidhalter Definitely worth another look at https://github.com/davidhalter/jedi/pull/829 . Optional activation would be most excellent (like a "use virtualenv in unsafe, but usable mode"). Thanks again @blueyed

[davidhalter]

@pappasam The Python API changes. So different Python versions have different C APIs. This leads to segfaults.

Virtualenv support is the next thing on my list (after the parser release in summer).

[pappasam]

@skamsie @blueyed I found a simple workaround, assuming you use Python 3.5 or 2.7 on Linux Mint (can't vouch for any other environment at this point):

" in your plugin list (assuming you use vim-plug):
Plug 'jmcantrell/vim-virtualenv'

" in your plugin constants configuration section
let g:virtualenv_auto_activate = 1

I confirm that 3.6 does not work yet; David Halter has already confirmed Python 3.6 development is pending other changes to core Jedi.

Hope this helps!

[WnP]

I've actually added this function in my ~/.zshrc:

function vim {
    PYTHONPATH=`python -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())"` /usr/bin/vim "$@"
}

modules installed in current virtualenv library have priority over the one vim is compiled with.

By the way, if vim is compiled with python 2.x and current virtualenv use python 3.x, any module from The Python Standard Library -like re- will be resolved from python 2.x library, and vice versa -if vim is compiled with python 3.

[davidhalter]

@WnP I wouldn't recommend this. You're screwing up everything by setting the PYTHONPATH.

[WnP]

@davidhalter everything like… what?

On my setup PYTHONPATH is not set at all by default

[davidhalter]

Well it's dangerous for the above reasons. Things might segfault with this. I'm just not recommending it to viewers of this thread.

[WnP]

@davidhalter ok, I see, thanks for the advice and all your work on vim-jedi

[cmcginty]

This script in .vimrc works for me:

py3 << EOF
import os, sys, pathlib
if 'VIRTUAL_ENV' in os.environ:
    venv = os.getenv('VIRTUAL_ENV')
    site_packages = next(pathlib.Path(venv, 'lib').glob('python*/site-packages'), None)
    if site_packages:
        sys.path.insert(0, str(site_packages))
EOF

[davidhalter]

@cmcginty I strongly discourage people from doing that. I know it works, but it can lead to segmentation faults and a lot of other issues.

Proper virtualenv support is coming, if you want to try it, it's part of jedi's master branch and especially venvs created with python -m venv should be working quite well.

[blueyed]

@davidhalter Do we need adjustments in jedi-vim for it? (IIRC deoplete-jedi needs some adjustments, since it uses removed internals of Jedi to improve the virtualenv handling)

[davidhalter]

Well, I currently run jedi-vim master with latest jedi and it works pretty well.

Eventually we want to adjust some things in jedi-vim to handle virtualenvs better. This is something we don't have to do, but it's just really easy. Scan for virtualenvs/Python environments every now and then and let the user choose which one he wants to use.

[blueyed]

Sounds good. Maybe a command similar to the force-python-version we have already.

Does Jedi has an API to return the known (virtual)envs?

[davidhalter]

See https://github.com/davidhalter/jedi/issues/1053.

[WnP]

ATM I use neovim with neovim (pep8 and yapf) python package(s) installed inside given virtual env.

With this configuration if I source my venv before opening neovim completion and doc are available from given virtual env.

I also use deoplete for completion and ale for syntax checking (both are async).

This setup is (at least for me) the perfect setup for python development using [neo]vim.

Check my .vimrc for configuration details

Let me know your through.

[xarthurx]

Activate virtualenv before opening neovim works. But using plugins like vim-virtualenv inside vim to activate the virtualenv doesn't work, unfortunately.

[cristobaltapia]

@xarthurx I use a simple hack to be able to get completions when changing the virtualenv inside vim. I do this by changing the g:deoplete#sources#jedi#python_path variable each time that the virtual environment is changes via VirtualEnvActivate (link to code). It is maybe not the best solution, but it works.

[aliceh75]

If you change the virtualenv from within vim using VirtualEnvActivate, you can trick jedi-vim into re-evaluating which Python it uses by setting let g:jedi#force_python_version="". This only works once - it's not a work around per se, but it shows jedi-vim is happy to change the Python version it uses at run time. Just a bit of plumbing is needed to allow users to reset the current Python version - presumably just clearing the cache holding the current python environment (maybe we could make JediClearCache do that? That would be fine for my use case)

[davidhalter]

When jedi-vim doesn't use a venv, it's probably because $VIRTUALENV is not set. What we should improve is selection of venvs, because for Jedi it's not a problem anyway. Jedi can deal with venvs very easily. So it's basically writing a good VIM UI for it.

I would not recommend using hacks like VirtualEnvActivate. This breaks Jedi potentially and other stuff. Jedi supports (virtual) environments natively.

[blueyed]

So it's basically writing a good VIM UI for it.

For reference: https://github.com/davidhalter/jedi-vim/pull/836

[davidhalter]

Virtualenvs are now supported very well. If you don't like the current support, we can discuss it. However I'm pretty sure there's not a lot that can be improved (there's also no open bugs in Jedi).

[bbliem]

When jedi-vim doesn't use a venv, it's probably because $VIRTUALENV is not set.

Thanks for pointing to that. As a note for posterity, it seems there was a typo and the environment variable is actually called VIRTUAL_ENV.