pip 9.0.1 tries to build wheels for caching even when wheel is not installed

[tardyp]

• Pip version: 9.0.1
• Python version: 3.5
• Operating system: debian unstable

Description:

User is reporting that buildbot is not installing under python3 unless wheel package is installed: https://github.com/buildbot/buildbot/issues/3495

What I've run:

Here are the commands I type:

$> pyvenv buildbot-sandbox
$> cd buildbot-sandbox/
$> . bin/activate
(buildbot-sandbox) $> pip install -U pip
Requirement already up-to-date: pip in ./lib/python3.5/site-packages
(buildbot-sandbox) $> pip --version
pip 9.0.1 from .../buildbot-sandbox/lib/python3.5/site-packages (python 3.5)
(buildbot-sandbox) $> pip install 'buildbot[bundle]'

I get the packages downloaded properly, but then most of the packages are failing like this (here the future package):

Building wheels for collected packages: future, Twisted, sqlalchemy, sqlalchemy-migrate, MarkupSafe, Tempita
  Running setup.py bdist_wheel for future ... error
  Complete output from command .../buildbot-sandbox/bin/python3.5 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-7r4e31_r/future/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpdhk60rxdpip-wheel- --python-tag cp35:
  usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: -c --help [cmd1 cmd2 ...]
     or: -c --help-commands
     or: -c cmd --help

  error: invalid command 'bdist_wheel'

  ----------------------------------------
  Failed building wheel for future

Then, when I redo the process but I take care of installing wheel before installing buildbot, everything is fine. Note, I simply install the wheel package like this:

(buildbot-sandbox) $> pip install wheel

[pradyunsg]

Hi! Thanks for filing this issue!

I am unable to reproduce this by creating a new virtualenv and uninstalling wheel in it. Could you please provide steps that don't use pyvenv and simply use virtualenv? :)

[tardyp]

@perror can you please answer?

[RonnyPfannschmidt]

i just took a look at the linked issue, pyenv is not virtualenv - to me it looks like pyenv has a bug and wont include what pip expects

also note that pip generally does just do a traditional install when wheel building fails, so its not clear whats broken there

[tardyp]

I just tried to repro using pyvenv instead of virtualenv on cloud9 ubuntu 14, and it does not build wheel cache for me buildbot[bundle] installs as expected.

could it be a transiant debian-unstable issue?

[pradyunsg]

I'm also unable to reproduce.

/tmp $ mkdir trial
/tmp $ cd trial
/tmp/trial $ pyvenv sandbox
WARNING: the pyenv script is deprecated in favour of `python3.6 -m venv`
/tmp/trial $ ls
sandbox
/tmp/trial $ cd sandbox
/tmp/trial/sandbox $ ls
bin             include         lib             pyvenv.cfg
/tmp/trial/sandbox $ ./bin/pip list
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
pip (9.0.1)
setuptools (28.8.0)
/tmp/trial/sandbox $ ./bin/pip install 'buildbot[bundle]'
Collecting buildbot[bundle]
  Using cached buildbot-0.9.10-py2.py3-none-any.whl
Collecting Twisted>=14.0.1 (from buildbot[bundle])
  Using cached Twisted-17.5.0.tar.bz2
Collecting sqlalchemy>=0.8.0 (from buildbot[bundle])
  Using cached SQLAlchemy-1.1.13.tar.gz
Collecting zope.interface>=4.1.1 (from buildbot[bundle])
  Using cached zope.interface-4.4.2-cp36-cp36m-macosx_10_6_intel.whl
Collecting future (from buildbot[bundle])
  Using cached future-0.16.0.tar.gz
Collecting Jinja2>=2.1 (from buildbot[bundle])
  Using cached Jinja2-2.9.6-py2.py3-none-any.whl
Collecting autobahn>=0.16.0 (from buildbot[bundle])
  Using cached autobahn-17.8.1-py2.py3-none-any.whl
Requirement already satisfied: setuptools>=8.0 in ./lib/python3.6/site-packages (from buildbot[bundle])
Collecting txaio>=2.2.2 (from buildbot[bundle])
  Using cached txaio-2.8.1-py2.py3-none-any.whl
Collecting PyJWT (from buildbot[bundle])
  Using cached PyJWT-1.5.2-py2.py3-none-any.whl
Collecting python-dateutil>=1.5 (from buildbot[bundle])
  Using cached python_dateutil-2.6.1-py2.py3-none-any.whl
Collecting sqlalchemy-migrate>=0.9 (from buildbot[bundle])
  Using cached sqlalchemy-migrate-0.11.0.tar.gz
Collecting buildbot-www==0.9.10; extra == "bundle" (from buildbot[bundle])
  Using cached buildbot_www-0.9.10-py2.py3-none-any.whl
Collecting buildbot-console-view==0.9.10; extra == "bundle" (from buildbot[bundle])
  Using cached buildbot_console_view-0.9.10-py2.py3-none-any.whl
Collecting buildbot-waterfall-view==0.9.10; extra == "bundle" (from buildbot[bundle])
  Using cached buildbot_waterfall_view-0.9.10-py2.py3-none-any.whl
Collecting buildbot-worker==0.9.10; extra == "bundle" (from buildbot[bundle])
  Using cached buildbot_worker-0.9.10-py2.py3-none-any.whl
Collecting constantly>=15.1 (from Twisted>=14.0.1->buildbot[bundle])
  Using cached constantly-15.1.0-py2.py3-none-any.whl
Collecting incremental>=16.10.1 (from Twisted>=14.0.1->buildbot[bundle])
  Using cached incremental-17.5.0-py2.py3-none-any.whl
Collecting Automat>=0.3.0 (from Twisted>=14.0.1->buildbot[bundle])
  Using cached Automat-0.6.0-py2.py3-none-any.whl
Collecting hyperlink>=17.1.1 (from Twisted>=14.0.1->buildbot[bundle])
  Using cached hyperlink-17.3.1-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23 (from Jinja2>=2.1->buildbot[bundle])
Collecting six>=1.10.0 (from autobahn>=0.16.0->buildbot[bundle])
  Using cached six-1.10.0-py2.py3-none-any.whl
Collecting pbr>=1.8 (from sqlalchemy-migrate>=0.9->buildbot[bundle])
  Using cached pbr-3.1.1-py2.py3-none-any.whl
Collecting decorator (from sqlalchemy-migrate>=0.9->buildbot[bundle])
  Using cached decorator-4.1.2-py2.py3-none-any.whl
Collecting sqlparse (from sqlalchemy-migrate>=0.9->buildbot[bundle])
  Using cached sqlparse-0.2.3-py2.py3-none-any.whl
Collecting Tempita>=0.4 (from sqlalchemy-migrate>=0.9->buildbot[bundle])
  Using cached Tempita-0.5.2.tar.gz
Collecting attrs (from Automat>=0.3.0->Twisted>=14.0.1->buildbot[bundle])
  Using cached attrs-17.2.0-py2.py3-none-any.whl
Installing collected packages: zope.interface, constantly, incremental, attrs, six, Automat, hyperlink, Twisted, sqlalchemy, future, MarkupSafe, Jinja2, txaio, autobahn, PyJWT, python-dateutil, pbr, decorator, sqlparse, Tempita, sqlalchemy-migrate, buildbot-www, buildbot-console-view, buildbot-waterfall-view, buildbot-worker, buildbot
  Running setup.py install for Twisted ... done
  Running setup.py install for sqlalchemy ... done
  Running setup.py install for future ... done
  Running setup.py install for Tempita ... done
  Running setup.py install for sqlalchemy-migrate ... done
Successfully installed Automat-0.6.0 Jinja2-2.9.6 MarkupSafe-1.0 PyJWT-1.5.2 Tempita-0.5.2 Twisted-17.5.0 attrs-17.2.0 autobahn-17.8.1 buildbot-0.9.10 buildbot-console-view-0.9.10 buildbot-waterfall-view-0.9.10 buildbot-worker-0.9.10 buildbot-www-0.9.10 constantly-15.1.0 decorator-4.1.2 future-0.16.0 hyperlink-17.3.1 incremental-17.5.0 pbr-3.1.1 python-dateutil-2.6.1 six-1.10.0 sqlalchemy-1.1.13 sqlalchemy-migrate-0.11.0 sqlparse-0.2.3 txaio-2.8.1 zope.interface-4.4.2

[perror]

Sorry, I was out for some time!

So, I tried again with virtualenv and got the exact same result. Here are the commands I used:

#> virtualenv --no-wheel --python=/usr/bin/python3.5 buildbot/
#> cd buildbot
#> . bin/activate
(buildbot) #> pip install 'buildbot[bundle]'

Also, here are the version of the software I am using:

(buildbot) #> pip --version
pip 9.0.1 from ..../buildbot/lib/python3.5/site-packages (python 3.5)
(buildbot) #> python --version
Python 3.5.4
(buildbot) #> virtualenv --version
15.1.0

Note that I also tried with Python 3.6 and I had the exact same problem!

[pradyunsg]

Thanks @perror!

I can't reproduce this on my OSX machine. I think this is something specific to your machine/system configuration... Could you run pip in verbose mode and paste it's output on a pastebin (or a GitHub Gist) and link to it here?

[perror]

Sure, here is the full log, I used Python 3.6 for this one (and I cleared pip cache).

install.txt

[pradyunsg]

huh.

@perror -- Does python -c "import wheel" succeed in that venv?

[perror]

Well, of course not because I specifically to install without wheel. But, I thought that wheel was not absolutely required for building these packages...

Anyway, here is the result of the command:

(buildbot) #> python -c "import wheel"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'wheel'

[pradyunsg]

Well, of course not

Yeah -- I just wanted to confirm that there wasn't something fishy happening with your paths... Still not sure what's causing this.

Also, it seems that pip is using cached data and you say that you've cleared the cache. :/

Could you please confirm that this still happens with the latest master (you can get it by doing pip install git+https://github.com/pypa/pip)?

(I'm still unable to figure out how to reproduce)

[perror]

Also, it seems that pip is using cached data and you say that you've cleared the cache. :/

Sorry, I thought the cache was stored in ~/.pip and it was finally in ~/.cache/pip. I purged it and rerun the whole thing. Here is the new log: install.txt

So, I tried to install pip within a venv built with the following command:

#> virtualenv --no-wheel --no-pip --python=/usr/bin/python3.6 buildbot
...
(buildbot) #> pip install git+https://github.com/pypa/pip

And, I got the following error (see log file attached).

install-pip.txt

[pradyunsg]

--no-pip

Don't create a virtualenv without pip. Please try it without this option. :)

Basically:

#> virtualenv --no-wheel --python=/usr/bin/python3.6 buildbot
...
(buildbot) #> pip install -I git+https://github.com/pypa/pip

[pradyunsg]

thought that wheel was not absolutely required for building these packages...

wheel package is needed to build wheels; not for installing from them. There's a conditional in the code that skips building wheels if wheel is not installed but for some reason that's not happening in your case...

The reason I'm asking you to try with the current master is that that conditional has changed (it is simpler now, less clauses) and this issue might have gotten fixed in the process.

[perror]

Sorry, I am discovering all this! :-) So, I re-ran the whole thing (without preventing pip to be present from start and the new version of pip got installed nicely) and here is the full log of the installation:

install-buildbot.txt

[pradyunsg]

Ah cool. So, that means that this issue is fixed in the current master. That makes me sort of happy. :P

Anyway, the exact compilation error you're facing is because you don't have the headers for the compilation available. That is a whole different problem and does not involve pip. None the less, if you're using the apt-get provided Python, you can just do apt install python3-dev and that should resolve it. :)

Here's an SO Question for that.

---

On that note, I think I can close this issue. :)

[pradyunsg]

Sorry, I am discovering all this! :-)

Aren't we all learning all the time? :wink:

[perror]

Well, thanks a lot @pradyunsg !

[pradyunsg]

Actually, I'll keep this one open for now and let one of @dstufft, @pfmoore or @xavfernandez close this if they don't think it'd be useful to debug why 9.0.1 failed. I certainly don't want to. :)

---

Well, thanks a lot @pradyunsg !

You're welcome! ^.^

[rjarry]

Hello,

after some painful digging, I may have found the root cause. This only happens with debian system packages (both virtualenv and pip).

pip has a debian patch that forces <sys.prefix>/usr/share/python-wheels/*.whl files to be prepended to sys.path: https://sources.debian.org/src/python-pip/9.0.1-2/debian/patches/debundle.patch/

On top of that, the --no-wheel option for virtualenv is taken into account, but all /usr/share/python-wheels/*.whl files are copied into <venv>/share/python-wheels (amongst them is wheel-0.29.0-py2.py3-none-any.whl). This is due to a debian patch: https://sources.debian.org/src/python-virtualenv/15.1.0+ds-1/debian/patches/use-wheels.patch/

All this results in this import:

https://github.com/pypa/pip/blob/2743edcc77fd660fa83d47090719e050c548a497/pip/commands/install.py#L10

not failing and therefore this:

https://github.com/pypa/pip/blob/2743edcc77fd660fa83d47090719e050c548a497/pip/commands/install.py#L335

being executed.

After which, the ./setup.py bdist_wheel command is executed in a subprocess:

https://github.com/pypa/pip/blob/2743edcc77fd660fa83d47090719e050c548a497/pip/wheel.py#L713

Since sys.path is not inherited by subprocesses, the <venv>/share/python-wheels/wheel-0.29.0-py2.py3-none-any.whl file is not in the python path which causes the following error:

error: invalid command 'bdist_wheel'

When virtualenv --no-wheel is specified, it should avoid copying wheel-*.whl to <venv>/share/python-wheels. This also applies to --no-pip and --no-setuptools options.

I'll report this to debian when I have some time.

[pradyunsg]

That's some great investigative work @rjarry!

That makes it certain that this is due to a debian patch. That explains why getting an unpatched pip from master fixed it and also why I couldn't reproduce it. :)

[pradyunsg]

@rjarry If you could, it would be nice if you link to the debian issue from here, just to tie loose ends. Whenever you find the time to. Thanks! 😬

[rjarry]

Here is the downstream bug https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=884353

In the meantime, the workaround is to disable caching pip install --no-cache-dir

[chrahunt]

It doesn't look like there's anything actionable from our side here, so I will close this issue.