Using python-environment with elpy

[sudarshang]

Currently elpy requires that the python libraries that it depends on (jedi, rope, yapf) be installed in every virtual environment. So if a user is working on 10 different projects she will need to install these libraries in each of the 10 different virtual environments. Could elpy use python-environment (https://github.com/tkf/emacs-python-environment) so that these libraries can all be installed when elpy is installed? This could make getting started with elpy easier.

[ChillarAnand]

Check if this helps https://github.com/jorgenschaefer/elpy/wiki/FAQ#q-do-i-have-to-install-all-these-packages-every-time

[sudarshang]

Thanks for pointing out the FAQ entry. I would like to avoid using 'togglesitepackages' if possible, as it takes away the isolation properties of virtualenv and we are back to issues such as 'works on my machine'.

[sudarshang]

Would elpy be interested in a pull request that implements this?

[jorgenschaefer]

Elpy uses pyvenv to interact with virtualenvs. I do not understand how using emacs-python-environment would improve the situation?

[sudarshang]

@jorgenschaefer thanks for responding!

All the python projects I have use a different virtualenv. These virtualenv are generated using tox. Currently using elpy requires that I install the necessary packages (jedi, yapf, etc) in each of the virtualenv or modify the generation of these virtualenv so that the use globally installed packages.

If elpy were to support emacs-python-environment then all these packages would need to be installed only once in that virtualenv. This also make elpy installation simpler for most users and allow each release of elpy to freeze the versions of these python packages. The elpy rpc process would be started in emacs python virtualenv. Pyvenv would still be used to pass in the virtualenv of the project associated with the current python buffer to the elpy rpc process.

Switching to emacs-python-enviroment involves changing not only the elisp to modify the launching of the elpy backend process but also the python code of the backend process so that the project specific virtualenv is currently handled by packages such as jedi. Please let me know if such a change is desirable and feasible.

[jorgenschaefer]

I still do not understand: What does emacs-python-environment different from pyvenv?

[vindarel]

I'll try to rephrase, to understand too:

• I have many python projects, all have their own venv. To run, say, a compile command, I must activate the corresponding venv. I do that with pyvenv.
• now I want to use elpy. So I need rope, jedi, etc, either installed in the venv of every of my python projects, or installed globally
• I don't want to install rope and co in every venv because 1) it's very annoying, my workflow is broken and 2) it pollutes my project's venv.
• elpy's faq says we can use virtualenvwrapper's togglesitepackages, so we can install rope and co globally only once. But
  • it's another little manual config for the user (undocumented)
  • it's risky isn't it ? I'm not sure anymore that my venv is complete, I may begin to use other system python packages that I should have in my venv, without noticing
  • => as I read, emacs-python-environnement fixes these. It installs rope and co in a separate venv from all others. When I switch of venv to work on my project with pyvenv, I will be able to use elpy, without taking care of installing rope and co in the venv or setting a virtualenvwrapper option.

Plus, emacs-python-environnement would allow elpy to freeze the versions of rope and co. (that seems awesome).

Plus, the project's readme also states a plugin can automate the installation of its python dependencies (looks great!).

[jorgenschaefer]

You still did not answer my question: What does emacs-python-environment do different from pyvenv.el, the module Elpy uses to support virtualenvs?

[sudarshang]

Sorry about the confusion.

emacs-python-environment does not support or manage project virtualenvs. It installs a python virtualenv in the users emacs.d. It also provides an easy way for lisp code to invoke python executables from the python virtualenv in emacs.d.

Please see https://github.com/tkf/emacs-jedi/blob/b6972af030416c57de6d045761d0ad6bccfdf07b/jedi-core.el#L1168

Do let me know if I should provide more info on emacs-python-enviroment.

[jorgenschaefer]

Ah, now I understand. Thank you.

It also provides an easy way for lisp code to invoke python executables from the python virtualenv in emacs.d.

Most of the modules Elpy uses need to have access to the modules of the project you are working on, i.e. need to be installed in the same virtualenv. How would that work with emacs-python-environment?

[sudarshang]

Most of the modules Elpy uses need to have access to the modules of the project you are working on, i.e. need to be installed in the same virtualenv. How would that work with emacs-python-environment?

I think the following modules

• jedi
• flake8
• yapf
• autopep8

will work even if they are not installed in the same virtualenv as the project. I have not used rope or importmagic so I do not know if they will need upstream fixes to include the project virtualenv. If you think that this enhancement (using emacs-python-env) is useful for elpy, I can start looking at that.

[jorgenschaefer]

importmagic definitely needs to be in the same virtualenv.

Last I checked, jedi does, too.

[vindarel]

Hi there, FYI, emacs-traad uses python-environment.

[aaron-kaplan]

importmagic definitely needs to be in the same virtualenv.

Last I checked, jedi does, too.

I have installed jedi in a venv of its own. I add that venv's bin dir to elpy-bin, and I add its site-packages dir to elpy-pythonpath. Then I can activate a different, project-specific venv, and both jedi and the project's dependencies will be available in the python interpreter. I haven't noticed anything yet that breaks because jedi isn't in the same virtualenv as the project's dependencies.

I like the idea suggested in this issue because it would make installation of elpy easier.

[galaunay]

Jedi does support that pretty well (now).

I am working on making Elpy works without having to install jedi (and others) in every virtualenvs (see PR #1548). Hope it will make everyone's life easier.

EDIT: Just saw your message in the PR, let's continue the discussion there.