search

bazelbuild / rules_python

Bazel Python Rules
https://rules-python.readthedocs.io
Apache License 2.0
530 stars 540 forks source link

Extras not being found/importable #1036

Closed snapbug closed 1 year ago

snapbug commented 1 year ago

🐞 bug report

Affected Rule

I'll say py_binary, because this is a runtime error, so will only manifest in executable targets. But might be further up/down the chain.

Is this a regression?

I don't know.

Description

Adding a lib with extras in requirements then trying to import those extras in a python file results in ModuleNotFoundError.

🔬 Minimal Reproduction

I created a repository here https://github.com/snapbug/bazel_python_extras

🔥 Exception or Error

When running the python_binary target:

Traceback (most recent call last):
  File "/private/var/tmp/_bazel_mcrane/13baa2ea46f1177a54f65afb9fe31f59/execroot/__main__/bazel-out/darwin-fastbuild/bin/foo.runfiles/__main__/foo.py", line 2, in 
    from ray import serve
ImportError: cannot import name 'serve' from 'ray' (/private/var/tmp/_bazel_mcrane/13baa2ea46f1177a54f65afb9fe31f59/execroot/__main__/bazel-out/darwin-fastbuild/bin/foo.runfiles/pip_ray_cpp/site-packages/ray/__init__.py)

where this import comes from an extra, and works with native python shell. The minimal repro has another example with a different error message.

🌍 Your Environment

Operating System:

OSX 13.1

Output of bazel version:

bazel 6.0.0

Rules_python version:

17.3

Anything else relevant?

I took a look at the other issues mentioning "extras", but they all seem to relate to older version of rules_python that required speif

linzhp commented 1 year ago

Same here, using something like requirement("ray[serve]") doesn't help either

linzhp commented 1 year ago

Actually, it worked after running bazel run //:py_deps.update.

snapbug commented 1 year ago

🤔 it doesn't seem to work for me no matter how many times I run //:py_deps.update.

I ran it on a linux box I have access to 

> uname -a
Linux starling 5.15.0-47-generic #51-Ubuntu SMP Thu Aug 11 07:51:15 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

and got a ModuleNotFoundError deep in the internals of the import of ray.serve, which feels like that might be rays problem, but at least it shows that the error is somewhere on the osx side, given everything else is the same. So some "progress" I guess...

linzhp commented 1 year ago

Yeah, I got ModuleNotFoundError: No module named 'colorama' on both Linux and macOS. So it's not rules_python not supporting extras, but specific to ray not handling its deps on colorama properly.

snapbug commented 1 year ago

Yeah, I think it's odd that I get different behaviour on os/linux, but either way I generally agree, not a rules_python issue. 🙂