search

microsoft / onnxruntime

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
https://onnxruntime.ai
MIT License
14.64k stars 2.92k forks source link

ModuleNotFoundError: No module named 'packaging' ERROR #13761

Closed IdoAmit198 closed 9 months ago

IdoAmit198 commented 1 year ago

Describe the issue

Hi everyone, I'm trying to clone onnxruntime repo and install (in order to later use the run_benchmark.sh script), but facing a really weird issue when I try to install. It errors on a missing package which I do have installed.

I already tried the solutions described here without a success.

Error and Output:

Processing /store/code/gpt2/quant/onnxruntime
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [17 lines of output]
      Traceback (most recent call last):
        File "/venv/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 351, in <module>
          main()
        File "/venv/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 333, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/venv/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/tmp/pip-build-env-31p956_z/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 338, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
        File "/tmp/pip-build-env-31p956_z/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 320, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-31p956_z/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 484, in run_setup
          super(_BuildMetaLegacyBackend,
        File "/tmp/pip-build-env-31p956_z/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 335, in run_setup
          exec(code, locals())
        File "<string>", line 17, in <module>
      ModuleNotFoundError: No module named 'packaging'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Please read To reproduce section to see what I did to have that error.

Expected behavior:

installation complete successfully.

I also attach my installed packages list:

installed_packages.txt

Would really appreciate any help on the topic!

To reproduce

I'm cloning the repo:

git clone https://github.com/microsoft/onnxruntime.git

I try to install:

pip install .

Then errors

And that's it, here I get the error you can see above in my description..

Urgency

No response

Platform

Linux

OS Version

20.04.4 LTS (Focal Fossa)

ONNX Runtime Installation

Other / Unknown

ONNX Runtime Version or Commit ID

onnxruntime Tag=1.13.1 , Commit ID=#13448

ONNX Runtime API

Python

Architecture

X86

Execution Provider

Other / Unknown

Execution Provider Library Version

No response

wangyems commented 1 year ago

please check if "packaging" exists in your python's site-packages

IdoAmit198 commented 1 year ago

Hi @wangyems , Thanks for the response. I checked and it exist in my python's site-packages.

hungrymonkey commented 1 year ago 
pip --version
pip 23.0.1 from /home/doof/gitrepo/dladata/venv/lib64/python3.10/site-packages/pip (python 3.10)
python3 --version
Python 3.10.10

pip install packaging
Requirement already satisfied: packaging in /home/doof/gitrepo/dladata/venv/lib/python3.10/site-packages (23.0)

 pip install git+https://github.com/microsoft/onnxruntime.git

 lsb_release  -a
LSB Version:    n/a
Distributor ID: openSUSE
Description:    openSUSE Tumbleweed
Release:    20230313
Codename:   n/a

Running command git clone --filter=blob:none --quiet https://github.com/microsoft/onnxruntime.git /tmp/pip-req-build-xy9dn4_d
  Resolved https://github.com/microsoft/onnxruntime.git to commit 5ac3a37be5d5bd25d167c2127b9c85f4b53668e7
  Running command git submodule update --init --recursive -q
j  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [17 lines of output]
      Traceback (most recent call last):
        File "/home/doof/gitrepo/dladata/venv/lib64/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/doof/gitrepo/dladata/venv/lib64/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/home/doof/gitrepo/dladata/venv/lib64/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/tmp/pip-build-env-w28k5dyg/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 338, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
        File "/tmp/pip-build-env-w28k5dyg/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 320, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-w28k5dyg/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 484, in run_setup
          super(_BuildMetaLegacyBackend,
        File "/tmp/pip-build-env-w28k5dyg/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 335, in run_setup
          exec(code, locals())
        File "<string>", line 17, in <module>
      ModuleNotFoundError: No module named 'packaging'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
pip list
Package                  Version
------------------------ -----------
aiohttp                  3.8.4
aiosignal                1.3.1
altair                   4.2.2
annoy                    1.17.1
anyio                    3.6.2
async-timeout            4.0.2
attrs                    22.2.0
beautifulsoup4           4.11.2
blinker                  1.5
cachetools               5.3.0
certifi                  2022.12.7
cffi                     1.15.1
charset-normalizer       3.1.0
click                    8.1.3
cmake                    3.26.0
coloredlogs              15.0.1
cryptography             39.0.2
datasets                 2.10.1
decorator                5.1.1
dill                     0.3.6
elastic-transport        8.4.0
elasticsearch            8.6.2
entrypoints              0.4
faiss-cpu                1.7.3
fastapi                  0.94.1
fasteners                0.18
fasttext                 0.9.2
filelock                 3.10.0
flatbuffers              23.3.3
fonttools                4.39.2
frozenlist               1.3.3
fsspec                   2023.3.0
gitdb                    4.0.10
GitPython                3.1.31
h11                      0.14.0
hnswlib                  0.7.0
huggingface-hub          0.13.2
humanfriendly            10.0
idna                     3.4
importlib-metadata       6.0.0
Jinja2                   3.1.2
joblib                   1.2.0
jsonschema               4.17.3
lit                      15.0.7
lxml                     4.9.2
lz4                      4.3.2
markdown-it-py           2.2.0
MarkupSafe               2.1.2
mdurl                    0.1.2
mpmath                   1.3.0
multidict                6.0.4
multiprocess             0.70.14
networkx                 3.0
nltk                     3.8.1
numpy                    1.24.2
nvidia-cublas-cu11       11.10.3.66
nvidia-cuda-cupti-cu11   11.7.101
nvidia-cuda-nvrtc-cu11   11.7.99
nvidia-cuda-runtime-cu11 11.7.99
nvidia-cudnn-cu11        8.5.0.96
nvidia-cufft-cu11        10.9.0.58
nvidia-curand-cu11       10.2.10.91
nvidia-cusolver-cu11     11.4.0.1
nvidia-cusparse-cu11     11.7.4.91
nvidia-nccl-cu11         2.14.3
nvidia-nvtx-cu11         11.7.91
onnxruntime              1.14.1
packaging                23.0
pandas                   1.5.3
paperai                  2.2.0
paperetl                 2.2.0
pdf-annotate             0.12.0
pdfminer.six             20221105
pdfrw                    0.4
Pillow                   9.4.0
pip                      23.0.1
protobuf                 3.20.3
pyarrow                  11.0.0
pybind11                 2.10.4
pycparser                2.21
pydantic                 1.10.6
pydeck                   0.8.0
Pygments                 2.14.0
pymagnitude-lite         0.1.143
Pympler                  1.0.1
pyrsistent               0.19.3
python-dateutil          2.8.2
pytz                     2022.7.1
pytz-deprecation-shim    0.1.0.post0
PyYAML                   6.0
regex                    2022.10.31
requests                 2.28.2
responses                0.18.0
rich                     13.3.2
rocketchat-async         1.0.1
scikit-learn             1.2.2
scipy                    1.10.1
semver                   2.13.0
sentence-transformers    2.2.2
sentencepiece            0.1.97
setuptools               65.5.0
six                      1.16.0
smmap                    5.0.0
sniffio                  1.3.0
soupsieve                2.4
starlette                0.26.1
streamlit                1.20.0
streamlit-chat           0.0.2.2
sympy                    1.11.1
text2digits              0.1.0
threadpoolctl            3.1.0
tokenizers               0.13.2
toml                     0.10.2
toolz                    0.12.0
torch                    2.0.0
torchvision              0.15.1
tornado                  6.2
tqdm                     4.65.0
transformers             4.27.1
triton                   2.0.0
txtai                    5.5.0
txtchat                  0.2.0
txtmarker                1.0.0
typing_extensions        4.5.0
tzdata                   2022.7
tzlocal                  4.2
urllib3                  1.26.15
uvicorn                  0.21.1
validators               0.20.0
watchdog                 2.3.1
websockets               10.4
wheel                    0.40.0
xxhash                   3.2.0
yarl                     1.8.2
zipp                     3.15.0

Same error here.

belm0 commented 1 year ago

I think this is a regression of onnxruntime 1.13

The setup.py now depends on 3rd party package packaging, without declaring the dependency. I believe it should be specified in pyproject.toml:

[build-system]
requires = ["packaging"]
bit-scientist commented 1 year ago

@belm0 putting packaging into pyproject.toml didn't help. I also tried installing the package with conda and apt-get. Neither worked. I wonder how one can handle this.

hieunct commented 1 year ago

Have

@belm0 putting packaging into pyproject.toml didn't help. I also tried installing the package with conda and apt-get. Neither worked. I wonder how one can handle this.

Have you found the solution for this? I have run into the same issue with Python3.10. Thanks!

saman-rahbar commented 1 year ago

Any update on this? I'm also getting the same error using python3.8. None of the other work around solutions helped. (E.g. including the setup file, pip installing the packaging library, etc...)

justinchuby commented 1 year ago

pip install . is not supported I don't think. I suggest following https://onnxruntime.ai/docs/build/inferencing.html#common-build-instructions to build and install the python wheel.

sinapy commented 1 year ago

I have the same issue when running pip install -v --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" --global-option="--deprecated_fused_adam" --global-option="--xentropy" --global-option="--fast_multihead_attn" ./

from fairseq https://github.com/facebookresearch/fairseq#requirements-and-installation

error:

  Running command Getting requirements to build wheel
  Traceback (most recent call last):
    File "C:\Users\301212298\source\repos\unilm\venv\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 363, in <module>
      main()
    File "C:\Users\301212298\source\repos\unilm\venv\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 345, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "C:\Users\301212298\source\repos\unilm\venv\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 130, in get_requires_for_build_wheel
      return hook(config_settings)
    File "C:\Users\301212~1\AppData\Local\Temp\pip-build-env-rckcnzxb\overlay\Lib\site-packages\setuptools\build_meta.py", line 341, in get_requires_for_build_wheel
      return self._get_build_requires(config_settings, requirements=['wheel'])
    File "C:\Users\301212~1\AppData\Local\Temp\pip-build-env-rckcnzxb\overlay\Lib\site-packages\setuptools\build_meta.py", line 323, in _get_build_requires
      self.run_setup()
    File "C:\Users\301212~1\AppData\Local\Temp\pip-build-env-rckcnzxb\overlay\Lib\site-packages\setuptools\build_meta.py", line 338, in run_setup
      exec(code, locals())
    File "<string>", line 5, in <module>
  ModuleNotFoundError: No module named 'packaging'
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: 'C:\Users\301212298\source\repos\unilm\venv\Scripts\python.exe' 'C:\Users\301212298\source\repos\unilm\venv\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py' get_requires_for_build_wheel 'C:\Users\301212~1\AppData\Local\Temp\tmpxjtt5ljt'
  cwd: C:\Users\301212298\source\repos\unilm\trocr\apex
  Getting requirements to build wheel ... error
error: subprocess-exited-with-error
erh94 commented 9 months ago

Any update on this, I also getting the issue with master

commit 9da3e36138dd24377fbb0b4022d891b3baf07b84

Steps followed : 

git clone https://github.com/microsoft/onnxruntime.git
cd onnxruntime
pip install -e .

Python 3.8.10

Error :

Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... error
  error: subprocess-exited-with-error

  × Getting requirements to build editable did not run successfully.
  │ exit code: 1
  ╰─> [19 lines of output]
      Traceback (most recent call last):
        File "/local/mnt/workspace/training/pipeline/env/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/local/mnt/workspace/training/pipeline/env/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/local/mnt/workspace/training/pipeline/env/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 132, in get_requires_for_build_editable
          return hook(config_settings)
        File "/tmp/pip-build-env-vkbqyp28/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 441, in get_requires_for_build_editable
          return self.get_requires_for_build_wheel(config_settings)
        File "/tmp/pip-build-env-vkbqyp28/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
        File "/tmp/pip-build-env-vkbqyp28/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-vkbqyp28/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 480, in run_setup
          super(_BuildMetaLegacyBackend, self).run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-vkbqyp28/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 18, in <module>
      ModuleNotFoundError: No module named 'packaging'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build editable did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
> pip show packaging
Name: packaging
Version: 23.2
Summary: Core utilities for Python packages
Home-page:
Author:
Author-email: Donald Stufft <donald@stufft.io>
License:
Location: /local/mnt/workspace/training/pipeline/env/lib/python3.8/site-packages
Requires:
Required-by: accelerate, datasets, docker, evaluate, gunicorn, huggingface-hub, ipykernel, jupyter_server, jupyterlab, jupyterlab_server, matplotlib, mlflow, nbconvert, onnxruntime-training, peft, qtconsole, QtPy, transformers

Thanks

dhivyeshrk commented 9 months ago

Any update on this? 

      Traceback (most recent call last):
        File "/opt/conda/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/opt/conda/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/opt/conda/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/tmp/pip-build-env-20pdylp3/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
        File "/tmp/pip-build-env-20pdylp3/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-20pdylp3/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 5, in <module>
      ModuleNotFoundError: No module named 'packaging'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

I have verified the installation of a non-conflicting version of 'packaging'. Thanks.

leeedwina430 commented 9 months ago

any update on this issue? I encountered the same one here

justinchuby commented 9 months ago

pip install . is not supported I don't think. I suggest following onnxruntime.ai/docs/build/inferencing.html#common-build-instructions to build and install the python wheel.

Please see the build instructions.

Long term, it would be really nice if we can support building directly from pip install . @snnn

tagMatrix4 commented 8 months ago

use: pip install -v --no-build-isolation -- no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext"

xty435768 commented 7 months ago

use: pip install -v --no-build-isolation -- no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext"

thanks! this solution works for me

roshanjoebenny commented 7 months ago

pip install -v --no-build-isolation -- no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" -> When I am executing this line I am getting an error thats given below

ERROR: Invalid requirement: '--global-option=--cpp_ext' Hint: = is not a valid operator. Did you mean == ?

angeloskanatas commented 7 months ago

pip install -v --no-build-isolation -- no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" -> When I am executing this line I am getting an error thats given below

ERROR: Invalid requirement: '--global-option=--cpp_ext' Hint: = is not a valid operator. Did you mean == ?

do: pip install -v --no-build-isolation --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./

wanan0414 commented 7 months ago

(swintssl) C:\Users\26804\apex>pip install -v --no-build-isolation --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./ Using pip 23.3.1 from E:\Anaconda\envs\swintssl\lib\site-packages\pip (python 3.8) DEPRECATION: --build-option and --global-option are deprecated. pip 24.0 will enforce this behaviour change. A possible replacement is to use --config-settings. Discussion can be found at https://github.com/pypa/pip/issues/11859 WARNING: Implying --no-binary=:all: due to the presence of --build-option / --global-option. Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple Processing c:\users\26804\apex Running command python setup.py egginfo Traceback (most recent call last): File "", line 2, in File "", line 34, in File "C:\Users\26804\apex\setup.py", line 132, in , bare_metal_version = get_cuda_bare_metal_version(CUDA_HOME) File "C:\Users\26804\apex\setup.py", line 19, in get_cuda_bare_metal_version raw_output = subprocess.check_output([cuda_dir + "/bin/nvcc", "-V"], universal_newlines=True) TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'

torch.version = 1.7.1

error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip. full command: 'E:\Anaconda\envs\swintssl\python.exe' -c ' exec(compile('"'"''"'"''"'"'

This is -- a caller that pip uses to run setup.py

#

- It imports setuptools before invoking setup.py, to enable projects that directly

import from distutils.core to work with newer packaging standards.

- It provides a clear error message when setuptools is not installed.

- It sets sys.argv[0] to the underlying setup.py, when invoking setup.py so

setuptools doesn'"'"'t think the script is -c. This avoids the following warning:

manifest_maker: standard file '"'"'-c'"'"' not found".

- It generates a shim setup.py, for handling setup.cfg-only projects.

import os, sys, tokenize

try: import setuptools except ImportError as error: print( "ERROR: Can not execute setup.py since setuptools is not available in " "the build environment.", file=sys.stderr, ) sys.exit(1)

file = %r sys.argv[0] = file

if os.path.exists(file): filename = file with tokenize.open(file) as f: setup_py_code = f.read() else: filename = "" setup_py_code = "from setuptools import setup; setup()"

exec(compile(setup_py_code, filename, "exec")) '"'"''"'"''"'"' % ('"'"'C:\Users\26804\apex\setup.py'"'"',), "", "exec"))' egg_info --egg-base 'C:\Users\26804\AppData\Local\Temp\pip-pip-egg-info-yxlkmtz1' cwd: C:\Users\26804\apex\ Preparing metadata (setup.py) ... error error: metadata-generation-failed

× Encountered error while generating package metadata. ╰─> See above for output.

note: This is an issue with the package mentioned above, not pip. hint: See above for details.

pip install -v --no-build-isolation -- no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" -> When I am executing this line I am getting an error thats given below ERROR: Invalid requirement: '--global-option=--cpp_ext' Hint: = is not a valid operator. Did you mean == ?

do: pip install -v --no-build-isolation --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./

roshanjoebenny commented 7 months ago

pip install -v --no-build-isolation --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./

I have executed above given command still I am getting the error

TypeError: unsupported operand type(s) for +: 'NoneType' and 'str' error: subprocess-exited-with-error

× Preparing metadata (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Preparing metadata (pyproject.toml) ... error error: metadata-generation-failed

× Encountered error while generating package metadata. ╰─> See above for output

Can anyone please help to solve this issue