Open jopollack opened 2 years ago
Thanks, that does seem odd! A
On Wed, Jun 29, 2022 at 8:29 AM jopollack @.***> wrote:
Rez Version 2.106.0
I'm not sure what's going on here, but when running rez-pip to install black, it leaves out one of the requirements, namely tomli. It installs tomli, but it doesn't add it to the package.py, so the resulting package is broken and I have to manually call tomli in the rez-env or add it into the package.py as a requirement.
rez-pip black --install -p /work/rez --python-version 3.7
Collecting black
Downloading https://files.pythonhosted.org/packages/2b/70/1d0e33a4df4ed73e9f02f698a29b5d94ff58e39f029c939ecf96a10fb1f3/black-22.6.0-py3-none-any.whl (156kB)
|████████████████████████████████| 163kB 25.6MB/s
Collecting typing-extensions>=3.10.0.0; python_version < "3.10" (from black)
Collecting typed-ast>=1.4.2; python_version < "3.8" and implementation_name == "cpython" (from black)
|████████████████████████████████| 849kB 34.4MB/s
Collecting mypy-extensions>=0.4.3 (from black)
Collecting pathspec>=0.9.0 (from black)
Collecting tomli>=1.1.0; python_full_version < "3.11.0a7" (from black)
Collecting platformdirs>=2 (from black)
Collecting click>=8.0.0 (from black)
Downloading https://files.pythonhosted.org/packages/c2/f1/df59e28c642d583f7dacffb1e0965d0e00b218e0186d7858ac5233dce840/click-8.1.3-py3-none-any.whl (96kB)
|████████████████████████████████| 102kB 10.4MB/s
Collecting importlib-metadata; python_version < "3.8" (from click>=8.0.0->black)
Collecting zipp>=0.5 (from importlib-metadata; python_version < "3.8"->click>=8.0.0->black)
Installing collected packages: typing-extensions, typed-ast, mypy-extensions, pathspec, tomli, platformdirs, zipp, importlib-metadata, click, black
Successfully installed black-22.6.0 click-8.1.3 importlib-metadata-4.12.0 mypy-extensions-0.4.3 pathspec-0.9.0 platformdirs-2.5.2 tomli-2.0.1 typed-ast-1.5.4 typing-extensions-4.2.0 zipp-3.8.0
WARNING: You are using pip version 19.2.3, however version 22.1.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
15:16:52 WARNING Metadata: missing: ['Home-page', 'Author'], warnings: []
15:16:52 WARNING Metadata: missing: ['Home-page', 'Author'], warnings: []
15:16:52 WARNING Metadata: missing: ['Home-page', 'Author'], warnings: []
15:16:52 INFO Installed [black-22.6.0] /work/rez/black/22.6.0/package.py (6e0a42ad67bd5da79eb94cd760314c7f48292a49)
15:16:52 INFO Installed [click-8.1.3] /work/rez/click/8.1.3/package.py (f98bb122ce29a25ef03cd5ba3ebdfa16f3a2d0c5)
15:16:52 INFO Installed [importlib_metadata-4.12.0] /work/rez/importlib_metadata/4.12.0/package.py (1ea170f09dad84a21d7e5918bcf56066e867f747)
15:16:52 INFO Installed [mypy_extensions-0.4.3] /work/rez/mypy_extensions/0.4.3/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
15:16:52 INFO Installed [pathspec-0.9.0] /work/rez/pathspec/0.9.0/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
15:16:53 INFO Installed [platformdirs-2.5.2] /work/rez/platformdirs/2.5.2/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
15:16:53 INFO Installed [tomli-2.0.1] /work/rez/tomli/2.0.1/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
15:16:53 INFO Installed [typed_ast-1.5.4] /work/rez/typed_ast/1.5.4/package.py (d9e9d80193dcd9578844ec4c2c22c9366ef0b88a)
15:16:53 INFO Installed [typing_extensions-4.2.0] /work/rez/typing_extensions/4.2.0/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
15:16:53 INFO Installed [zipp-3.8.0] /work/rez/zipp/3.8.0/package.py (88a70aca30cb79a278872594adf043dc6c40af99)
15:16:53 INFO 10 packages were installed.
-- coding: utf-8 --
name = 'black'
version = '22.6.0'
description = 'The uncompromising code formatter.'
authors = ['Łukasz Langa @.***']
tools = [
'black', 'blackd'
]
requires = [
'click-8.0.0+', 'platformdirs-2+', 'mypy_extensions-0.4.3+', 'pathspec-0.9.0+'
]
variants = [['python-3.7', 'typing_extensions-3.10.0.0+', 'typed_ast-1.4.2+']]
def commands():
env.PYTHONPATH.append('{root}/python') env.PATH.append('{root}/bin')
help = [['Home Page', 'https://github.com/psf/black']]
timestamp = 1656454612
hashed_variants = True
is_pure_python = True
from_pip = True
pip_name = 'black (22.6.0)'
format_version = 2
Traceback (most recent call last):
File "/work/rez/black/22.6.0/6e0a42ad67bd5da79eb94cd760314c7f48292a49/bin/black", line 6, in
from black import patched_main
File "/work/rez/black/22.6.0/6e0a42ad67bd5da79eb94cd760314c7f48292a49/python/black/init.py", line 53, in
from black.files import (
File "/work/rez/black/22.6.0/6e0a42ad67bd5da79eb94cd760314c7f48292a49/python/black/files.py", line 31, in
import tomli as tomllib
ModuleNotFoundError: No module named 'tomli'
As you can see, it installs tomli, but it doesn't end up in the package.py, so attempting to run black causes an import error.
— Reply to this email directly, view it on GitHub https://github.com/AcademySoftwareFoundation/rez/issues/1341, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMOUSS5G5CCV3XI2EP2ZK3VRN4FPANCNFSM52DW52GA . You are receiving this because you are subscribed to this thread.Message ID: @.***>
My guess is that's because it is a conditional requirement and something likely breaks:
Requires-Dist: tomli (>=1.1.0) ; python_full_version < "3.11.0a7"
(the difference to the other requirements besides the alpha version is python_full_version
instead of python_version
Also: Works fine on windows. It ends up in the variant as it is version specific:
name = 'black'
version = '22.6.0'
description = 'The uncompromising code formatter.'
authors = ['Łukasz Langa lukasz@langa.pl']
tools = [
'black.exe',
'blackd.exe'
]
requires = [
'platformdirs-2+',
'pathspec-0.9.0+',
'click-8.0.0+',
'mypy_extensions-0.4.3+'
]
variants = \
[['platform-windows',
'arch-AMD64',
'python-3.7',
'typed_ast-1.4.2+',
'typing_extensions-3.10.0.0+',
'tomli-1.1.0+']]
We do support python_full_version
, see https://github.com/AcademySoftwareFoundation/rez/blob/master/src/rez/utils/pip.py#L391. And as you can see, we use the packaging library to evaluate the markers (which is the library also used by pip and plenty of other tools).
So we would have to debug that issue and see what's going on as it's supposed to work.
What distro are you on? As i can not repro it on windows
FWIW I reproduced this on Windows 3 days ago when the ticket was made but tried again last night and couldn't reproduce it. The only difference between then and now is that I added a python 3 Rez package via @instinct-vfx 's rezpy. Before that, pip was getting found from Rez's own python. That may be a contributing factor.
Yep, just confirmed again. I removed the python 3 packages that I'd installed via rezpy, Rez used its own python (rez-python --version
== "3.7.9"), and then this happened.
variants = \
[['platform-windows',
'arch-AMD64',
'python-3.7',
'typing_extensions-3.10.0.0+',
'typed_ast-1.4.2+']]
Note the missing tomli.
I then reinstalled a python Rez package (3.7.9) using rezpy
rezpy --python_version=3.7.9
Then re-ran rez-pip and got these variants
variants = \
[['platform-windows',
'arch-AMD64',
'python-3.7',
'typed_ast-1.4.2+',
'tomli-1.1.0+',
'typing_extensions-3.10.0.0+']]
I suspect @instinct-vfx uses a python package via rezpy, which is why he cannot reproduce it at the moment. But if you remove the python package and use Rez's own python, it should happen.
In my case I discovered the bug on Linux using rez-pip in a default environment with rez 2.106.0, so the existing pip or python packages rez comes with may be a factor.
Your original reproduction doesn't show which python / pip was used (I'd thought rez-pip always outputs that at the start. Did you omit that information or was it truly not shown in your output?) but yes, likely Rez's own rez-python
was chosen or a python package that may not be set up properly. In my personal experience, in Windows, rez-bind
fails to build a proper Python package and then rez-pip
detects that and falls back to rez-python
.
Yes, I omitted it. It found pip-19.2.3 in the python-3.7 I had installed.
rez-pip black --install -p /work/rez --python-version 3.7
15:16:46 INFO Trying to use pip from python package
15:16:48 INFO Found pip-19.2.3 inside /work/rez/python/3.7.7/package.py. Will use it via /work/rez/python/3.7.7/bin/python3.7
15:16:48 INFO Installing 'black' with pip taken from '/work/rez/python/3.7.7/bin/python3.7'
Rez Version 2.106.0
I'm not sure what's going on here, but when running rez-pip to install black, it leaves out one of the requirements, namely tomli. It installs tomli, but it doesn't add it to the package.py, so the resulting package is broken and I have to manually call tomli in the rez-env or add it into the package.py as a requirement.
As you can see, it installs tomli, but it doesn't end up in the package.py, so attempting to run black causes an import error.