unable to build from source - `cannot import name 'packaging' from 'pkg_resources'`

forresti commented 2 months ago

If you do not know the root cause of the problem / bug, and wish someone to help you, please post according to this template:

🐛 Bugs / Unexpected behaviors

I am on Linux, and I am unable to build the pytorch3d code from source.

Instructions To Reproduce the Issue:

Please include the following (depending on what the issue is):

Any changes you made (git diff) or code you wrote I did not modify the code.

The exact command(s) you ran:


conda create -n my_env
conda activate my_env

conda install pytorch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 pytorch-cuda=11.7 -c pytorch -c nvidia

git clone https://github.com/facebookresearch/pytorch3d.git cd pytorch3d pip install -e .


3. What you observed (including the full logs):

Obtaining file:///home/ubuntu/SyncMVD/pytorch3d
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [8 lines of output]
Traceback (most recent call last):
File "", line 2, in
File "", line 34, in
File "/home/ubuntu/SyncMVD/pytorch3d/setup.py", line 17, in
from torch.utils.cpp_extension import CppExtension, CUDA_HOME, CUDAExtension
File "/home/ubuntu/anaconda3/envs/syncmvd2/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 25, in from pkg_resources import packaging # type: ignore[attr-defined]
ImportError: cannot import name 'packaging' from 'pkg_resources' (/home/ubuntu/anaconda3/envs/syncmvd2/lib/python3.8/site-packages/pkg_resources/init.py) [end of output]



The problem seems to happen when I use that older version of pytorch.
When I install pytorch with `conda install pytorch::pytorch torchvision torchaudio -c pytorch`, then pytorch3d builds fine. But, I am using this old PyTorch because I am trying to setup https://github.com/LIU-Yuxin/SyncMVD, which uses this old PyTorch.

bottler commented 2 months ago

Can you update setuptools in the environment?

Fanie-Visagie commented 2 months ago

I have the exact same issue...I have upgraded and downgraded setup tools, package manager, cuda, etc etc....

(models) D:\03 Innovation\pytorch3d>pip install -v -e .
Using pip 24.1.2 from C:\Users\perso\AppData\Roaming\Python\Python310\site-packages\pip (python 3.10)
Obtaining file:///D:/03%20Innovation/pytorch3d
  Running command python setup.py egg_info
  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "D:\03 Innovation\pytorch3d\setup.py", line 17, in <module>
      from torch.utils.cpp_extension import CppExtension, CUDA_HOME, CUDAExtension
    File "C:\ProgramData\miniforge3\envs\models\lib\site-packages\torch\utils\cpp_extension.py", line 25, in <module>
      from pkg_resources import packaging  # type: ignore[attr-defined]
  ImportError: cannot import name 'packaging' from 'pkg_resources' (C:\ProgramData\miniforge3\envs\models\lib\site-packages\pkg_resources\__init__.py)
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: 'C:\ProgramData\miniforge3\envs\models\python.exe' -c '
  exec(compile('"'"''"'"''"'"'
  # This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
  #
  # - It imports setuptools before invoking setup.py, to enable projects that directly
  #   import from `distutils.core` to work with newer packaging standards.
  # - It provides a clear error message when setuptools is not installed.
  # - It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so
  #   setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:
  #     manifest_maker: standard file '"'"'-c'"'"' not found".
  # - It generates a shim setup.py, for handling setup.cfg-only projects.
  import os, sys, tokenize

  try:
      import setuptools
  except ImportError as error:
      print(
          "ERROR: Can not execute `setup.py` since setuptools is not available in "
          "the build environment.",
          file=sys.stderr,
      )
      sys.exit(1)

  __file__ = %r
  sys.argv[0] = __file__

  if os.path.exists(__file__):
      filename = __file__
      with tokenize.open(__file__) as f:
          setup_py_code = f.read()
  else:
      filename = "<auto-generated setuptools caller>"
      setup_py_code = "from setuptools import setup; setup()"

  exec(compile(setup_py_code, filename, "exec"))
  '"'"''"'"''"'"' % ('"'"'D:\\03 Innovation\\pytorch3d\\setup.py'"'"',), "<pip-setuptools-caller>", "exec"))' egg_info --egg-base 'C:\Users\perso\AppData\Local\Temp\pip-pip-egg-info-p6rqbll5'
  cwd: D:\03 Innovation\pytorch3d\\
  Preparing metadata (setup.py) ... error
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

bottler commented 2 months ago

@Fanie-Visagie I reduced your sample to the main output. I think we can't help - this looks like a general question of versions of python, setuptools and pytorch. It looks like you are on Windows, which may distinguish it from OP.

misrolox commented 2 months ago

Hello I have the same problem trying to build a docker image to run https://github.com/NVlabs/BundleSDF.git. The docker image is this one: dockerfile. It is failing at installing https://github.com/facebookresearch/pytorch3d.git with the same error as OP:

=> ERROR [13/18] RUN conda init bash && echo "conda activate py38" >> ~/.bashrc && conda activate py38 && pip install torch==1.11.0+cu113 torchvision==0.12.0+cu113 torchaudio==0.11.0 -- 737.3s

[13/18] RUN conda init bash && echo "conda activate py38" >> ~/.bashrc && conda activate py38 && pip install torch==1.11.0+cu113 torchvision==0.12.0+cu113 torchaudio==0.11.0 --extra-index-url https://download.pytorch.org/whl/cu113 && pip install "git+https://github.com/facebookresearch/pytorch3d.git@stable" && pip install trimesh opencv-python wandb matplotlib imageio tqdm open3d ruamel.yaml sacred kornia pymongo pyrender jupyterlab ninja && conda install -y -c anaconda scipy:
0.252 no change /opt/conda/condabin/conda
0.252 no change /opt/conda/bin/conda
0.252 no change /opt/conda/bin/conda-env 0.252 no change /opt/conda/bin/activate 0.252 no change /opt/conda/bin/deactivate 0.252 no change /opt/conda/etc/profile.d/conda.sh 0.252 no change /opt/conda/etc/fish/conf.d/conda.fish 0.252 no change /opt/conda/shell/condabin/Conda.psm1 0.252 no change /opt/conda/shell/condabin/conda-hook.ps1 0.252 no change /opt/conda/lib/python3.12/site-packages/xontrib/conda.xsh 0.252 no change /opt/conda/etc/profile.d/conda.csh 0.252 modified /root/.bashrc 0.252 0.252 ==> For changes to take effect, close and re-open your current shell. <== 0.252 0.531 Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/cu113 1.573 Collecting torch==1.11.0+cu113 2.072 Downloading https://download.pytorch.org/whl/cu113/torch-1.11.0%2Bcu113-cp38-cp38-linux_x86_64.whl (1637.0 MB) 683.2 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.6/1.6 GB 3.5 MB/s eta 0:00:00 685.3 Collecting torchvision==0.12.0+cu113 685.4 Downloading https://download.pytorch.org/whl/cu113/torchvision-0.12.0%2Bcu113-cp38-cp38-linux_x86_64.whl (22.3 MB) 689.3 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 22.3/22.3 MB 5.4 MB/s eta 0:00:00 690.0 Collecting torchaudio==0.11.0 690.0 Downloading https://download.pytorch.org/whl/cu113/torchaudio-0.11.0%2Bcu113-cp38-cp38-linux_x86_64.whl (2.9 MB) 690.5 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.9/2.9 MB 6.0 MB/s eta 0:00:00 691.0 Collecting typing-extensions (from torch==1.11.0+cu113) 691.3 Downloading typing_extensions-4.12.2-py3-none-any.whl.metadata (3.0 kB) 692.2 Collecting numpy (from torchvision==0.12.0+cu113) 692.2 Downloading numpy-1.24.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.6 kB) 692.7 Collecting requests (from torchvision==0.12.0+cu113) 692.8 Downloading requests-2.32.3-py3-none-any.whl.metadata (4.6 kB) 693.6 Collecting pillow!=8.3.*,>=5.3.0 (from torchvision==0.12.0+cu113) 693.7 Downloading pillow-10.4.0-cp38-cp38-manylinux_2_28_x86_64.whl.metadata (9.2 kB) 694.4 Collecting charset-normalizer<4,>=2 (from requests->torchvision==0.12.0+cu113) 694.5 Downloading charset_normalizer-3.3.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (33 kB) 695.1 Collecting idna<4,>=2.5 (from requests->torchvision==0.12.0+cu113) 695.2 Downloading idna-3.7-py3-none-any.whl.metadata (9.9 kB) 696.1 Collecting urllib3<3,>=1.21.1 (from requests->torchvision==0.12.0+cu113) 696.1 Downloading urllib3-2.2.2-py3-none-any.whl.metadata (6.4 kB) 696.8 Collecting certifi>=2017.4.17 (from requests->torchvision==0.12.0+cu113) 696.8 Downloading certifi-2024.7.4-py3-none-any.whl.metadata (2.2 kB) 696.9 Downloading pillow-10.4.0-cp38-cp38-manylinux_2_28_x86_64.whl (4.5 MB) 700.5 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.5/4.5 MB 1.3 MB/s eta 0:00:00 700.5 Downloading numpy-1.24.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB) 710.2 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 17.3/17.3 MB 2.3 MB/s eta 0:00:00 710.2 Downloading requests-2.32.3-py3-none-any.whl (64 kB) 710.3 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 64.9/64.9 kB 3.0 MB/s eta 0:00:00 710.3 Downloading typing_extensions-4.12.2-py3-none-any.whl (37 kB) 710.4 Downloading certifi-2024.7.4-py3-none-any.whl (162 kB) 710.4 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 163.0/163.0 kB 2.8 MB/s eta 0:00:00 710.5 Downloading charset_normalizer-3.3.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (141 kB) 710.5 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 141.1/141.1 kB 2.6 MB/s eta 0:00:00 710.6 Downloading idna-3.7-py3-none-any.whl (66 kB) 710.6 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 66.8/66.8 kB 2.4 MB/s eta 0:00:00 710.6 Downloading urllib3-2.2.2-py3-none-any.whl (121 kB) 710.7 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 121.4/121.4 kB 3.3 MB/s eta 0:00:00 710.8 Installing collected packages: urllib3, typing-extensions, pillow, numpy, idna, charset-normalizer, certifi, torch, requests, torchvision, torchaudio 728.1 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv 728.1 Successfully installed certifi-2024.7.4 charset-normalizer-3.3.2 idna-3.7 numpy-1.24.4 pillow-10.4.0 requests-2.32.3 torch-1.11.0+cu113 torchaudio-0.11.0+cu113 torchvision-0.12.0+cu113 typing-extensions-4.12.2 urllib3-2.2.2 729.6 Collecting git+https://github.com/facebookresearch/pytorch3d.git@stable 729.6 Cloning https://github.com/facebookresearch/pytorch3d.git (to revision stable) to /tmp/pip-req-build-ltyxrweo 729.6 Running command git clone --filter=blob:none --quiet https://github.com/facebookresearch/pytorch3d.git /tmp/pip-req-build-ltyxrweo 735.8 Running command git checkout -q 89653419d0973396f3eff1a381ba09a07fffc2ed 736.5 Resolved https://github.com/facebookresearch/pytorch3d.git to commit 89653419d0973396f3eff1a381ba09a07fffc2ed 736.5 Preparing metadata (setup.py): started 737.1 Preparing metadata (setup.py): finished with status 'error' 737.1 error: subprocess-exited-with-error 737.1
737.1 × python setup.py egg_info did not run successfully. 737.1 │ exit code: 1 737.1 ╰─> [8 lines of output] 737.1 Traceback (most recent call last): 737.1 File "", line 2, in 737.1 File "", line 34, in 737.1 File "/tmp/pip-req-build-ltyxrweo/setup.py", line 17, in 737.1 from torch.utils.cpp_extension import CppExtension, CUDA_HOME, CUDAExtension 737.1 File "/opt/conda/envs/py38/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 23, in 737.1 from pkg_resources import packaging # type: ignore[attr-defined] 737.1 ImportError: cannot import name 'packaging' from 'pkg_resources' (/opt/conda/envs/py38/lib/python3.8/site-packages/pkg_resources/init.py) 737.1 [end of output] 737.1
737.1 note: This error originates from a subprocess, and is likely not a problem with pip. 737.1 error: metadata-generation-failed 737.1 737.1 × Encountered error while generating package metadata. 737.1 ╰─> See above for output. 737.1 737.1 note: This is an issue with the package mentioned above, not pip. 737.1 hint: See above for details.

3 warnings found (use --debug to expand):

LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 72)

LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 87)

LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 88) dockerfile:75

74 |
75 | >>> RUN conda init bash &&\ 76 | >>> echo "conda activate py38" >> ~/.bashrc &&\ 77 | >>> conda activate py38 &&\ 78 | >>> pip install torch==1.11.0+cu113 torchvision==0.12.0+cu113 torchaudio==0.11.0 --extra-index-url https://download.pytorch.org/whl/cu113 &&\ 79 | >>> pip install "git+https://github.com/facebookresearch/pytorch3d.git@stable" &&\ 80 | >>> pip install trimesh opencv-python wandb matplotlib imageio tqdm open3d ruamel.yaml sacred kornia pymongo pyrender jupyterlab ninja &&\ 81 | >>> conda install -y -c anaconda scipy 82 |

ERROR: failed to solve: process "/bin/bash --login -c conda init bash && echo \"conda activate py38\" >> ~/.bashrc && conda activate py38 && pip install torch==1.11.0+cu113 torchvision==0.12.0+cu113 torchaudio==0.11.0 --extra-index-url https://download.pytorch.org/whl/cu113 && pip install \"git+https://github.com/facebookresearch/pytorch3d.git@stable\" && pip install trimesh opencv-python wandb matplotlib imageio tqdm open3d ruamel.yaml sacred kornia pymongo pyrender jupyterlab ninja && conda install -y -c anaconda scipy" did not complete successfully: exit code: 1

bottler commented 2 months ago

@misrolox BTW, you will need to have fvcore and iopath installed before the pip install will work, but that isn't the actual problem here. I don't know how to solve your error. Can you try with newer versions of python and pytorch?

misrolox commented 2 months ago

Thank you for your answer. My problem was fixed by adding the line pip install setuptools==69.5.1 as mentioned by this issue.

facebookresearch / pytorch3d

unable to build from source - `cannot import name 'packaging' from 'pkg_resources'` #1834

🐛 Bugs / Unexpected behaviors

Instructions To Reproduce the Issue:

LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 88) dockerfile:75