iiasa / message_ix

The integrated assessment and energy systems model MESSAGEix
https://docs.messageix.org
Apache License 2.0
111 stars 149 forks source link

Update install instructions with note about using conda and pip together #835

Closed glatterf42 closed 4 weeks ago

glatterf42 commented 2 months ago

Recently, I helped a colleague resolve installation issues involving conda and pip. The error message they received was (not too sure about the error name):

AttributeError: 'message_ix' has no attribute 'Scenario'

I know I've seen these before and I don't know if the cause is always the same. If I remember correctly, I usually advised people to install everything from scratch in a new venv, which also solved the problem. This time, however, I understood the issue in more detail: The colleague had used conda to install message_ix, but now wanted to utilize a specific branch, so tried to upgrade to an installation from source. Of course, pip and conda don't mix well, but in theory, using pip after conda and then sticking with pip is fine in conda-managed venvs. However, the user had a global pip installation (referring to the global Python interpreter, that is), but none in their conda-venv. So they tried running pip install and it succeeded, but not in the way they wanted to. We were able to solve this problem by creating a new venv with conda, using conda to install pip in there, and then only using pip to install the message stack.

So my suggestion here is to include a note about similar issues in the 'full' installation guide once we manage to separate 'full' and 'quick'. It should go something like this (although it could be expanded to cover the install from pypi, too):

.. note::
   Installing from source requires using ``pip``. We strongly recommend installing in a virtual environment (venv), so for these steps, we strongly recommend using a venv managing tool that is pip-compatible like `python's venv <https://docs.python.org/3/library/venv.html>`__, `pipenv <https://pipenv.pypa.io/en/latest/>`__, or `virtualenv <https://virtualenv.pypa.io/en/latest/user_guide.html>`__, to name just a few.

   If you insist on using ``conda`` to mange your venv, please read `conda's guide to using pip in a venv <https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#using-pip-in-an-environment>`__. In particular, please make sure you use ``conda`` only to install ``pip`` in your venv and then use that specific ``pip`` for all further install commands.
khaeru commented 2 months ago
AttributeError: 'message_ix' has no attribute 'Scenario'

I know I've seen these before and I don't know if the cause is always the same.

AFAICT, this happens in the following case:

What happens in this case is that the editable install files are created in their site-packages/ directly, but an empty directory site-packages/message_ix/ is left behind from the regular install. IOW, pip removes the files but not the directory for the existing, regular install.

When the user next tries to import message_ix, Python finds the empty directory site-pacakges/message_ix/ and imports it, but this contains absolutely nothing, hence the API is not usable.

The fix I've tried is to first pip uninstall message-ix before pip install --editable .. This has worked in all cases I've encountered (regardless of whether users are using conda, venv, etc. or not).

glatterf42 commented 2 months ago

Great, thanks! We can include this possible fix for the issue in a Known issue with the same PR :)