Unable to install tools like pipenv: 'Did not find any scripts. Uninstalling.'

[jambonrose]

I'm new to pipsi, and installed it using the curl command. I'm running macOS 10.13.6.

$ python3.7 -V
Python 3.7.0

$ pip -V
pip 18.0 from /opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pip (python 3.7)

$ curl https://raw.githubusercontent.com/mitsuhiko/pipsi/master/get-pipsi.py | python3.7
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  6577  100  6577    0     0  41891      0 --:--:-- --:--:-- --:--:-- 41891
Installing pipsi
sys.executable=/opt/local/Library/Frameworks/Python.framework/Versions/3.7/bin/python3.7 sys.real_prefix=None executable=/opt/local/Library/Frameworks/Python.framework/Versions/3.7/bin/python3.7
Collecting pipsi
  Using cached https://files.pythonhosted.org/packages/4e/16/297caa009ac37bb52a31ed8a76ee50fe373794f32a6ec7a227d7af65e15b/pipsi-0.9.tar.gz
Collecting Click (from pipsi)
  Using cached https://files.pythonhosted.org/packages/34/c1/8806f99713ddb993c5366c362b2f908f18269f8d792aff1abfd700775a77/click-6.7-py2.py3-none-any.whl
Collecting virtualenv (from pipsi)
  Using cached https://files.pythonhosted.org/packages/b6/30/96a02b2287098b23b875bc8c2f58071c35d2efe84f747b64d523721dc2b5/virtualenv-16.0.0-py2.py3-none-any.whl
Installing collected packages: Click, virtualenv, pipsi
  Running setup.py install for pipsi ... done
Successfully installed Click-6.7 pipsi-0.9 virtualenv-16.0.0
You are using pip version 10.0.1, however version 18.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Installed pipsi binary in /Users/andrew/.local/bin
pipsi is now installed.

I'm a bit confused as to how I ended up with pip 10.0.1 in the pipsi environment given the existing installation of 18.0, but hey, no big deal.

$ pipsi --version
pipsi, version 0.9, python /Users/andrew/.local/venvs/pipsi/bin/python3.7

$ /Users/andrew/.local/venvs/pipsi/bin/python3.7 -m pip -V
pip 10.0.1 from /Users/andrew/.local/venvs/pipsi/lib/python3.7/site-packages/pip (python 3.7)

$ /Users/andrew/.local/venvs/pipsi/bin/python3.7 -m pip install -U pip
Collecting pip
  Using cached https://files.pythonhosted.org/packages/5f/25/e52d3f31441505a5f3af41213346e5b6c221c9e086a166f3703d2ddaf940/pip-18.0-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 10.0.1
    Uninstalling pip-10.0.1:
      Successfully uninstalled pip-10.0.1
Successfully installed pip-18.0

$ /Users/andrew/.local/venvs/pipsi/bin/python3.7 -m pip -V
pip 18.0 from /Users/andrew/.local/venvs/pipsi/lib/python3.7/site-packages/pip (python 3.7)

I also take a moment to upgrade setuptools.

$ /Users/andrew/.local/venvs/pipsi/bin/python3.7 -m pip install -U setuptools
Collecting setuptools
  Using cached https://files.pythonhosted.org/packages/ff/f4/385715ccc461885f3cedf57a41ae3c12b5fec3f35cce4c8706b1a112a133/setuptools-40.0.0-py2.py3-none-any.whl
Installing collected packages: setuptools
  Found existing installation: setuptools 39.0.1
    Uninstalling setuptools-39.0.1:
      Successfully uninstalled setuptools-39.0.1
Successfully installed setuptools-40.0.0

I make sure that /Users/andrew/.local/bin is always in my $PATH, and I'm off to the races.

$ pipsi install pew
Running virtualenv with interpreter /Users/andrew/.local/venvs/pipsi/bin/python3.7
Using base prefix '/opt/local/Library/Frameworks/Python.framework/Versions/3.7'
/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/virtualenv.py:1041: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
New python executable in /Users/andrew/.local/venvs/pew/bin/python3.7
Also creating executable in /Users/andrew/.local/venvs/pew/bin/python
Installing setuptools, pip, wheel...done.
Collecting pew
  Using cached https://files.pythonhosted.org/packages/53/77/9ca8e2b4eb0c5d0de534f8de64a5e6dbe8375e7134aa9797e8f9ce7d6f53/pew-1.1.5-py2.py3-none-any.whl
Requirement already satisfied: virtualenv-clone>=0.2.5 in ./.local/venvs/pipsi/lib/python3.7/site-packages (from pew) (0.3.0)
Requirement already satisfied: setuptools>=17.1 in ./.local/venvs/pipsi/lib/python3.7/site-packages (from pew) (39.0.1)
Requirement already satisfied: virtualenv>=1.11 in ./.local/venvs/pipsi/lib/python3.7/site-packages (from pew) (16.0.0)
Installing collected packages: pew
Successfully installed pew-1.1.5
Did not find any scripts.  Uninstalling.

$ pipsi install pipenv
Running virtualenv with interpreter /Users/andrew/.local/venvs/pipsi/bin/python3.7
Using base prefix '/opt/local/Library/Frameworks/Python.framework/Versions/3.7'
/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/virtualenv.py:1041: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
New python executable in /Users/andrew/.local/venvs/pipenv/bin/python3.7
Also creating executable in /Users/andrew/.local/venvs/pipenv/bin/python
Installing setuptools, pip, wheel...done.
Collecting pipenv
  Using cached https://files.pythonhosted.org/packages/eb/64/9b2747d54f2008ac3dfe86c0b1c8ec126042726fd8a540d5208d26732701/pipenv-2018.7.1-py3-none-any.whl
Requirement already satisfied: setuptools>=36.2.1 in ./.local/venvs/pipsi/lib/python3.7/site-packages (from pipenv) (39.0.1)
Requirement already satisfied: pip>=9.0.1 in ./.local/venvs/pipsi/lib/python3.7/site-packages (from pipenv) (18.0)
Requirement already satisfied: virtualenv in ./.local/venvs/pipsi/lib/python3.7/site-packages (from pipenv) (16.0.0)
Collecting certifi (from pipenv)
  Using cached https://files.pythonhosted.org/packages/7c/e6/92ad559b7192d846975fc916b65f667c7b8c3a32bea7372340bfe9a15fa5/certifi-2018.4.16-py2.py3-none-any.whl
Collecting virtualenv-clone>=0.2.5 (from pipenv)
  Using cached https://files.pythonhosted.org/packages/6d/c2/dccb5ccf599e0c5d1eea6acbd058af7a71384f9740179db67a9182a24798/virtualenv_clone-0.3.0-py2.py3-none-any.whl
Installing collected packages: certifi, virtualenv-clone, pipenv
Successfully installed certifi-2018.4.16 pipenv-2018.7.1 virtualenv-clone-0.3.0
Did not find any scripts.  Uninstalling.

Oh dear. This is the same error as #154 (and the other problems linked in that issue), except the conditions appear to be different.

I note that there's something a little odd going on here, though, as trying to install with Python 3.6 gets me funny errors about pip.

$ pipsi install --python=`which python3.6` pew
Running virtualenv with interpreter /opt/local/bin/python3.6
Using base prefix '/opt/local/Library/Frameworks/Python.framework/Versions/3.6'
New python executable in /Users/andrew/.local/venvs/pew/bin/python3.6
Also creating executable in /Users/andrew/.local/venvs/pew/bin/python
Installing setuptools, pip, wheel...done.
Traceback (most recent call last):
  File "/Users/andrew/.local/venvs/pew/bin/pip", line 7, in <module>
    from pip._internal import main
ModuleNotFoundError: No module named 'pip'
Failed to pip install.  Aborting.

$ python3.6 -m pip -V
pip 18.0 from /opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip (python 3.6)

I'd love help if this looks like a misconfig. Alternatively, if we think this might be a bug, I'm happy to help track it down and open a PR to fix the issue.

Thanks in advance.

[jambonrose]

Aha. After taking some time to look through issues, PRs and some code, I see that I need to be installing from the master branch.

For others who may be new to this:

$ export PIPSI_DEBUG=True

$ curl https://raw.githubusercontent.com/mitsuhiko/pipsi/master/get-pipsi.py | python3.7 - --src 'git+https://github.com/mitsuhiko/pipsi.git#egg=pipsi'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  6577  100  6577    0     0  43556      0 --:--:-- --:--:-- --:--:-- 44140
Installing pipsi
sys.executable=/opt/local/Library/Frameworks/Python.framework/Versions/3.7/bin/python3.7 sys.real_prefix=None executable=/opt/local/Library/Frameworks/Python.framework/Versions/3.7/bin/python3.7
Collecting pipsi from git+https://github.com/mitsuhiko/pipsi.git#egg=pipsi
  Cloning https://github.com/mitsuhiko/pipsi.git to /private/var/folders/3p/6d4nmfcx3h31f96yjtr7szlr0000gn/T/pip-install-32cngvjv/pipsi
Collecting Click (from pipsi)
  Using cached https://files.pythonhosted.org/packages/34/c1/8806f99713ddb993c5366c362b2f908f18269f8d792aff1abfd700775a77/click-6.7-py2.py3-none-any.whl
Collecting virtualenv (from pipsi)
  Using cached https://files.pythonhosted.org/packages/b6/30/96a02b2287098b23b875bc8c2f58071c35d2efe84f747b64d523721dc2b5/virtualenv-16.0.0-py2.py3-none-any.whl
Installing collected packages: Click, virtualenv, pipsi
  Running setup.py install for pipsi ... done
Successfully installed Click-6.7 pipsi-0.10.dev0 virtualenv-16.0.0
You are using pip version 10.0.1, however version 18.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Installed pipsi binary in /Users/andrew/.local/bin
pipsi is now installed.

$ pipsi --version
pipsi, version 0.10.dev0, python /Users/andrew/.local/venvs/pipsi/bin/python3.7

$ /Users/andrew/.local/venvs/pipsi/bin/python3.7 -m pip -V
pip 10.0.1 from /Users/andrew/.local/venvs/pipsi/lib/python3.7/site-packages/pip (python 3.7)

$ /Users/andrew/.local/venvs/pipsi/bin/python3.7 -m pip install -U pip setuptools
Collecting pip
  Using cached https://files.pythonhosted.org/packages/5f/25/e52d3f31441505a5f3af41213346e5b6c221c9e086a166f3703d2ddaf940/pip-18.0-py2.py3-none-any.whl
Collecting setuptools
  Using cached https://files.pythonhosted.org/packages/ff/f4/385715ccc461885f3cedf57a41ae3c12b5fec3f35cce4c8706b1a112a133/setuptools-40.0.0-py2.py3-none-any.whl
Installing collected packages: pip, setuptools
  Found existing installation: pip 10.0.1
    Uninstalling pip-10.0.1:
      Successfully uninstalled pip-10.0.1
  Found existing installation: setuptools 39.0.1
    Uninstalling setuptools-39.0.1:
      Successfully uninstalled setuptools-39.0.1
Successfully installed pip-18.0 setuptools-40.0.0

$ pipsi install pew
python: /Users/andrew/.local/venvs/pipsi/bin/python3.7, python_bin_semver: (3, 7, 0)
get_real_python run ['/Users/andrew/.local/venvs/pipsi/bin/python3.7', '-c', 'import sys; print("{},{}".format(getattr(sys, "real_prefix", ""), sys.version_info.major))']: 0, ,3, 
Popen: ['/Users/andrew/.local/venvs/pipsi/bin/python3.7', '-m', 'venv', '/Users/andrew/.local/venvs/pew']
Popen: ['/Users/andrew/.local/venvs/pew/bin/python', '-m', 'pip', 'install', 'pew']
Collecting pew
  Using cached https://files.pythonhosted.org/packages/53/77/9ca8e2b4eb0c5d0de534f8de64a5e6dbe8375e7134aa9797e8f9ce7d6f53/pew-1.1.5-py2.py3-none-any.whl
Requirement already satisfied: setuptools>=17.1 in ./.local/venvs/pew/lib/python3.7/site-packages (from pew) (39.0.1)
Collecting virtualenv>=1.11 (from pew)
  Using cached https://files.pythonhosted.org/packages/b6/30/96a02b2287098b23b875bc8c2f58071c35d2efe84f747b64d523721dc2b5/virtualenv-16.0.0-py2.py3-none-any.whl
Collecting virtualenv-clone>=0.2.5 (from pew)
  Using cached https://files.pythonhosted.org/packages/6d/c2/dccb5ccf599e0c5d1eea6acbd058af7a71384f9740179db67a9182a24798/virtualenv_clone-0.3.0-py2.py3-none-any.whl
Installing collected packages: virtualenv, virtualenv-clone, pew
Successfully installed pew-1.1.5 virtualenv-16.0.0 virtualenv-clone-0.3.0
You are using pip version 10.0.1, however version 18.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
  Linked script /Users/andrew/.local/bin/pew
Done.

$ which pew
/Users/andrew/.local/bin/pew

Still totally baffled as to what's going on with pip.

$ python3.7 -m pip -V
pip 18.0 from /opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pip (python 3.7)

$ pip -V
pip 18.0 from /opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pip (python 3.7)

[jambonrose]

The pip error appears to be the fault of my setup, as using venv directly causes a new virtual environment with the wrong pip version.

$ python -V
Python 3.7.0

$ python -m venv venv_test

$ ./venv_test/bin/pip -V
pip 10.0.1 from /Users/andrew/venv_test/lib/python3.7/site-packages/pip (python 3.7)

$ ./venv_test/bin/pip3 -V
pip 10.0.1 from /Users/andrew/venv_test/lib/python3.7/site-packages/pip (python 3.7)

I'm therefore closing this issue, as a PR already exists to fix the docs.

[jambonrose]

For posterity: that is actually how venv is supposed to work! I had misunderstood the intent wrt installing pip.

[floatingpurr]

Thanks @jambonrose for your hints. I came across the same issue. In particular, I'm using pyenv with Python 3.7.0 set as global interpreter.

$ pyenv versions
system
2.7.15
* 3.7.0 (set by /Users/foo/.pyenv/version)

$ pip --version
pip 18.1 from /Users/foo/.pyenv/versions/3.7.0/lib/python3.7/site-packages/pip (python 3.7)

I installed pipsi with curl:

curl https://raw.githubusercontent.com/mitsuhiko/pipsi/master/get-pipsi.py | python

And I got

You are using pip version 10.0.1, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

So, If I understand well, that's not a problem but it's the normal behavior of the virtualenvs created in ~/.local/venvs carrying by default the old version of pip. Or better, that is the pip version coming with Python 3.7.0 in the brand new virtualenv. Am I right?

[jambonrose]

Glad that helped!

My (limited) understanding is as follows (corrections welcome).

Python's venv may optionally setup pip, and when it does so it calls ensurepip. At the moment, Python's ensurepip bundles pip v10 (as a wheel!), which is why new environments are loaded with that version of pip.

Short tangent I thought was interesting:

When I first looked into this, I got really confused by the upgrade flag being passed to ensurepip. The documentation states first that the upgrade parameter is used to "ensure the installed version of pip is at least as recent as the one bundled with ensurepip" but later states that the flag "indicates whether or not to upgrade an existing installation of an earlier version of pip to the bundled version." But that's not how it's being used! It's being passed directly to pip! Turns out that the --no-index and --find-links options focus pip into using just the packages being bundled, meaning PyPI will never be queried.

[floatingpurr]

Got it! Thank you once again. Therefore, if I am not mistaken, there is no way to install (CLI) packages with pipsi using your "global" pip version. Anyway, it should not be a real problem.