pip - ImportError: cannot import name main

[rlipscombe]

I used asdf global python system; I have pip installed in ~/.local:

$ pip -V
pip 19.0.1 from /home/roger/.local/lib/python2.7/site-packages/pip (python 2.7)

But when I change to a directory which uses a local python version (2.7.15), pip no longer works:

$ cd src/foo
$ pip -V
Traceback (most recent call last):
  File "/home/roger/.asdf/installs/python/2.7.15/bin/pip", line 7, in <module>
    from pip import main
ImportError: cannot import name main

[rlipscombe]

If I set up a python virtualenv, it fixes pip, but I was kinda hoping to avoid doing that (or for asdf-python to do it transparently).

For reference, I set up the virtualenv as follows:

virtualenv -p $(asdf where python)/bin/python .virtualenv
. .virtualenv/bin/activate

[oxitnik]

This happens because python uses pip module from user site-packages folder.

~/my/foo$ python -m site
sys.path = [
    '/home/dmitry/my/foo',
    '/home/dmitry/.asdf/installs/python/2.7.15/lib/python27.zip',
    '/home/dmitry/.asdf/installs/python/2.7.15/lib/python2.7',
    '/home/dmitry/.asdf/installs/python/2.7.15/lib/python2.7/plat-linux2',
    '/home/dmitry/.asdf/installs/python/2.7.15/lib/python2.7/lib-tk',
    '/home/dmitry/.asdf/installs/python/2.7.15/lib/python2.7/lib-old',
    '/home/dmitry/.asdf/installs/python/2.7.15/lib/python2.7/lib-dynload',
    '/home/dmitry/.local/lib/python2.7/site-packages',
    '/home/dmitry/.asdf/installs/python/2.7.15/lib/python2.7/site-packages',
]
USER_BASE: '/home/dmitry/.local' (exists)
USER_SITE: '/home/dmitry/.local/lib/python2.7/site-packages' (exists)
ENABLE_USER_SITE: True

As you can see, user site-packages (/home/dmitry/.local/lib/python2.7/site-packages) is before (/home/dmitry/.asdf/installs/python/2.7.15/lib/python2.7/site-packages), so python looks for pip module there first. Is is possible to disable adding site-packages folder to sys.path using -s option or PYTHONNOUSERSITE=1 environment variable:

~/my/foo$ python -s -m site
sys.path = [
    '/home/dmitry/my/foo',
    '/home/dmitry/.asdf/installs/python/2.7.15/lib/python27.zip',
    '/home/dmitry/.asdf/installs/python/2.7.15/lib/python2.7',
    '/home/dmitry/.asdf/installs/python/2.7.15/lib/python2.7/plat-linux2',
    '/home/dmitry/.asdf/installs/python/2.7.15/lib/python2.7/lib-tk',
    '/home/dmitry/.asdf/installs/python/2.7.15/lib/python2.7/lib-old',
    '/home/dmitry/.asdf/installs/python/2.7.15/lib/python2.7/lib-dynload',
    '/home/dmitry/.asdf/installs/python/2.7.15/lib/python2.7/site-packages',
]
USER_BASE: '/home/dmitry/.local' (exists)
USER_SITE: '/home/dmitry/.local/lib/python2.7/site-packages' (exists)
ENABLE_USER_SITE: False

As you can see, there is no /home/dmitry/.local/lib/python2.7/site-packages in sys.path.

Same with pip:

~/my/foo$ python -m pip -V                
pip 19.0.2 from /home/dmitry/.local/lib/python2.7/site-packages/pip (python 2.7)

~/my/foo$ python -s -m pip -V
pip 9.0.3 from /home/dmitry/.asdf/installs/python/2.7.15/lib/python2.7/site-packages (python 2.7)

So as workaround to your issue, instead of pip you can use:

python -s -m pip

[danhper]

I never had this issue so I'm not really sure how to go with this. What should be done on asdf-python side to fix this?

[delucca]

Same thing happens here with asdf and pipx

If I run python -s -m pip it works, but, in my opinion, asdf-python should handle this...

[CaseyLabs]

For users coming here from a Google search for asdf python picking up wrong pip install location, or searching for asdf python .local/lib site-packages:

Issue Description

• Install a Python package using system pip (e.g. pip3 install ansible)

• Next, install Python via asdf (asdf plugin add python; asdf install python 3.8.12)

• Now, install the same pip package under asdf:

. $HOME/.asdf/asdf.sh
asdf global python 3.8.12
pip3 install ansible
asdf reshim python

• asdf will fail to install the package under its .asdf/shims directory, as it still sees the package installed under $HOME/.local/lib/python3.8/site-packages. Because of this, you'll be unable to use the installed Ansible package using asdf in this example.

Why? Because sys.path for Python is picking up the .local/site-packages before the .asdf/installs/.../site-packages:

$ python -m site

sys.path = [
'/home/username',
    '/home/username/.asdf/installs/python/3.8.12/lib/python38.zip',
    '/home/username/.asdf/installs/python/3.8.12/lib/python3.8',
    '/home/username/.asdf/installs/python/3.8.12/lib/python3.8/lib-dynload',
    '/home/username/.local/lib/python3.8/site-packages',
    '/home/username/.asdf/installs/python/3.8.12/lib/python3.8/site-packages',
]

SOLUTION

(Thanks to @oxitnik for figuring this one out!)

The PYTHONNOUSERSITE environment variable can be used to disable user site-packages to sys.path. Update your ~/.bashrc or ~/.zshrc to use:

. $HOME/.asdf/asdf.sh && PYTHONNOUSERSITE=1

And your issue will be solved!

cc @danhper

[jxie2016]

[jxie@vmromangdps tests]$ python test_RelativeFluxCalib.py Traceback (most recent call last): File "test_RelativeFluxCalib.py", line 25, in import asdf ImportError: No module named asdf