search

tubular / rules_pygen

Rules for generating native Bazel Python libraries from requirements.txt
Apache License 2.0
15 stars 1 forks source link

error: invalid command 'bdist_wheel' #4

Open thundergolfer opened 5 years ago

thundergolfer commented 5 years ago

Ran this within our Bazel repo and it failed at the following place: https://github.com/tubular/rules_pygen/blob/bdbf28f4bc9fe52010e28fd8d1cc919fdf569dda/src/rules_pygen/rules_generator.py#L364.

The error message was saying Failed to build wheel X for a number of packages listed in our requirements.txt (eg. pyyaml, httppretty, jsondiff, ...). I looked into the source and ran the command this tool was running like so: 

python3.6 -m pip wheel --disable-pip-version-check --requirement third_party/python/requirements.txt --wheel-dir /tmp/blah

This gave more detailed error messages, one of which being this:

  Running setup.py clean for stringcase
  Building wheel for wrapt (setup.py) ... error
  Complete output from command /nix/store/f698bw0zrv79vzihz6xwcyd84sncrqp8-python3-3.6.8/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/private/var/folders/0b/p1vldpsd4f99nc50bv4t4csh0000gn/T/pip-wheel-qejpjlkq/wrapt/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /private/var/folders/0b/p1vldpsd4f99nc50bv4t4csh0000gn/T/pip-wheel-mwqsnd8h:
  usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: -c --help [cmd1 cmd2 ...]
     or: -c --help-commands
     or: -c cmd --help

  error: invalid command 'bdist_wheel'

This typically indicates a dependency on wheel (thus needing to do pip install wheel). I will investigate this further, but wanted to throw this issue up now in case you recognise immediately what the problem is.

Btw, thanks for open-sourcing this! It's much better than the hacky alternative I wrote at our company. 👍

c4urself commented 5 years ago

I'm curious as to what version of pip you're using: python3.6 -m pip --version.

Not 100% up-to-date with the goings on in pip-land but I believe you fix this in two ways: 1) upgrade your pip (wheel is a subdependency) python3.6 -m pip install --upgrade pip 2) directly install wheel like you suggested; python3.6 -m pip install wheel

The weird thing is that that pip does know about the wheel subcommand in your case but the setuptools wheel plugin clearly doesn't work which is strange...

I'd be open to add an additional check that makes sure pip has the necessary minimum version to work properly

thundergolfer commented 5 years ago

python3.6 -m pip --version: pip 19.0.3 from /nix/store/w8wb057378iyxids2lsh6sk6w5n5rrdv-python3.6-pip-19.0.3/lib/python3.6/site-packages/pip (python 3.6)

thundergolfer commented 5 years ago

Option 2 fixes the issue. Option 1 outputs this: 

    raise PyBazelRuleGeneratorException("Pip call caused an error: {}".format(err))
rules_pygen.rules_generator.PyBazelRuleGeneratorException: Pip call caused an error:   Failed building wheel for avro-python3
  Failed building wheel for aws-sam-translator
  Failed building wheel for future
  Failed building wheel for httpretty
  Failed building wheel for jsondiff
  Failed building wheel for pycparser
  Failed building wheel for pygithub
  Failed building wheel for pyhocon
  Failed building wheel for pyrsistent
  Failed building wheel for pyyaml
  Failed building wheel for stringcase
  Failed building wheel for wrapt
ERROR: Failed to build one or more wheels

which is the same as before.

With option 2, I get a different error: 

Processing wheelinfo for /var/folders/0b/p1vldpsd4f99nc50bv4t4csh0000gn/T/tmpf01qd6hu/avro_python3-1.9.0-py3-none-any.whl
DEBUG:rules_pygen.rules_generator:Wheel name is: avro-python3
DEBUG:rules_pygen.rules_generator:Will find additional wheels for other platforms using prefix: avro_python3-1.9.0
Traceback (most recent call last):
  File "/private/var/tmp/_bazel_jbelotti/a8a1f81a1422a72c79c6c81641a8c816/execroot/data_science/bazel-out/darwin-fastbuild/bin/external/rules_pygen/generator.runfiles/rules_pygen/src/rules_pygen/__main__.py", line 41, in <module>
    gen.run()
  File "/private/var/tmp/_bazel_jbelotti/a8a1f81a1422a72c79c6c81641a8c816/execroot/data_science/bazel-out/darwin-fastbuild/bin/external/rules_pygen/generator.runfiles/rules_pygen/src/rules_pygen/rules_generator.py", line 289, in run
    deps = self._parse_wheel_dependencies(wheel_links)
  File "/private/var/tmp/_bazel_jbelotti/a8a1f81a1422a72c79c6c81641a8c816/execroot/data_science/bazel-out/darwin-fastbuild/bin/external/rules_pygen/generator.runfiles/rules_pygen/src/rules_pygen/rules_generator.py", line 424, in _parse_wheel_dependencies
    raise PyBazelRuleGeneratorException("Dependency {} is missing wheels!".format(dependency))
rules_pygen.rules_generator.PyBazelRuleGeneratorException: Dependency avro_python3 is missing wheels!

So that seems like progress.