Failed to run `` pip install -e . --no-deps --isolated '' to install the local git repo of pip.

[hongyi-zhao]

Hi,

I pulled the master version of pip and then try to install it into my conda environment with --isolated option which previously works but now fails. See the following:

 (aiida) werner@ubuntu-01:~/.repo/github.com/pypa/pip.git$ pip install -e . --no-deps --isolated 
Obtaining file:///home/werner/.repo/github.com/pypa/pip.git
  Installing build dependencies ... error
  ERROR: Command errored out with exit status 3:
   command: /home/werner/.pyenv/versions/anaconda3-2020.02/envs/aiida/bin/python /home/werner/.pyenv/versions/anaconda3-2020.02/envs/aiida/lib/python3.7/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-c5u1fgfi/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel
       cwd: None
  Complete output (1 lines):
  ERROR: Could not find an activated virtualenv (required).
  ----------------------------------------
ERROR: Command errored out with exit status 3: /home/werner/.pyenv/versions/anaconda3-2020.02/envs/aiida/bin/python /home/werner/.pyenv/versions/anaconda3-2020.02/envs/aiida/lib/python3.7/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-c5u1fgfi/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel Check the logs for full command output.

Any hints?

Regards

[deveshks]

This error comes from https://github.com/pypa/pip/blob/445711c533b707f056a8c9512d55aa2986dee157/src/pip/_internal/cli/base_command.py#L172-L178

By any chance, did you set --require-virtualenv, or --require-venv, maybe as pip config, or via a environment variable?

[hongyi-zhao]

By any chance, did you set -require-virtualenv, or --require-venv, maybe as pip config, or via a environment variable?

I set it as follows:

$ grep require-virtualenv ~/.pip/pip.conf 
require-virtualenv = true

But, as you can see, I also use the --isolated option which will ignore environment variables and user configuration. Why still it will fail?

Regards.

[deveshks]

So I tried both pip install -e . --no-deps --isolated, and pip install . --no-deps --isolated outside a virtualenv after setting PIP_REQUIRE_VIRTUALENV env var to True, and they both fail.

$ pip install . --no-deps --isolated 
Processing /Users/devesh/pip
  Installing build dependencies ... error
  ERROR: Command errored out with exit status 3:
   command: /Users/devesh/.pyenv/versions/3.8.2/bin/python3.8 /Users/devesh/.pyenv/versions/3.8.2/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /private/var/folders/xg/blp845_s0xn093dyrtgy936h0000gp/T/pip-build-env-sacyege7/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel
       cwd: None
  Complete output (1 lines):
  ERROR: Could not find an activated virtualenv (required).
  ----------------------------------------
ERROR: Command errored out with exit status 3: /Users/devesh/.pyenv/versions/3.8.2/bin/python3.8 /Users/devesh/.pyenv/versions/3.8.2/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /private/var/folders/xg/blp845_s0xn093dyrtgy936h0000gp/T/pip-build-env-sacyege7/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel Check the logs for full command output.

But explicitly giving a package name works.

$ pip install six --no-deps --isolated 
Collecting six
  Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Installing collected packages: six
Successfully installed six-1.15.0

[hongyi-zhao]

Strange. Is pip intentionally designed to work in this way? Anyway, it seems like a bug.

[deveshks]

Strange. Is pip intentionally designed to work in this way? If not, it seems like a bug.

I am not sure about that, but I will wait for others to chime in before making that assumption.

[uranusjr]

This sounds like a bug. The difference between six and . as displayed above is the former case uses a wheel, while the latter builds from source with PEP 517. I can reproduce the same error by adding --no-binary (and --force-reinstall since otherwise pip would just use the already-installed version):

pip install --force-reinstall --isolated --no-deps --no-binary=:all: pip

I believe the problem is that we’re not correctly unsetting the PIP_REQUIRE_VIRTUALENV environment variable when running the isolated build environment. We probably should do that since the isolated environment is, well, isolated, and it probably does not matter whether a virtual environment is used? Also the virtual environment requirement was already checked before any PEP 517 build happens anyway, so persuambly the requirement is already correctly matched if an installation reach the PEP 517 part.

/cc @pfmoore

[pfmoore]

That sounds like a reasonable analysis, although I'm not very familiar with the build isolation code (technically it's from the PEP 518 work, which I didn't do).

On the other hand, should PIP_REQUIRE_VIRTUALENV be getting checked for a --prefix install (which is what the isolation code uses) in any case? If you're specifying a --prefix you're not actually installing in the virtual environment or the system environment.

[uranusjr]

On the other hand, should PIP_REQUIRE_VIRTUALENV be getting checked for a --prefix install (which is what the isolation code uses) in any case? If you're specifying a --prefix you're not actually installing in the virtual environment or the system environment.

Good point, and perhaps the same applies for --target as well. I guess this is like a lot of situations in pip, nobody really thought the myriads of flag combinations though when they are introduced.

Who should I ask for expertise in this area (build isolation)?

[pfmoore]

The contributor who wrote that code has since disappeared, so I doubt you're going to find anyone with a better idea than me (or any of the other pip maintainers) 🙁

I'd be fine with unsetting PIP_REQUIRE_VIRTUALENV in the isolated environment code (I assume we already disable any config file that might also set --require-virtualenv?) as a straightforward fix. But looking at the interaction of --prefix (and, as you say, --target) and --require-virtualenv might be a better long-term approach.