daquexian / onnx-simplifier

Simplify your onnx model
Apache License 2.0
3.87k stars 383 forks source link

[BUG] Cannot install on Jetson Xavier AGX #214

Open GiorgosBetsos opened 2 years ago

GiorgosBetsos commented 2 years ago

Describe the bug Steps to reproduce on a Jetson Xavier AGX device:

I have created a new conda env on my Jetson Xavier AGX custom board:

$ conda create -n yolov5 python=3.8

I activated the env and installed onnx using conda:

$ conda activate yolov5
$ conda install onnx

Then I tried to install onnx-simplifier:

$ pip3 install -U pip && pip3 install onnxsim

Installation fails. I get following output:

/usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead from cryptography.utils import int_from_bytes /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead from cryptography.utils import int_from_bytes Defaulting to user installation because normal site-packages is not writeable Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com Requirement already satisfied: pip in /home/nvidia/.local/lib/python3.6/site-packages (21.3.1) /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead from cryptography.utils import int_from_bytes /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead from cryptography.utils import int_from_bytes Defaulting to user installation because normal site-packages is not writeable Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com Collecting onnxsim Downloading onnxsim-0.4.5.tar.gz (20.0 MB) |████████████████████████████████| 20.0 MB 11.3 MB/s Preparing metadata (setup.py) ... done WARNING: Discarding https://files.pythonhosted.org/packages/ad/d4/bb94a3e3c4eeb2b97ea4ac42f5dfa71ee239150bea50c49602b2e6bb3e44/onnxsim-0.4.5.tar.gz#sha256=d691ffe81f0e1b1b4f40fbb18aca7d802e8dd3cc032db128f0e78bfa5f0caa59 (from https://pypi.org/simple/onnxsim/) (requires-python:>=3.6). Requested onnxsim from https://files.pythonhosted.org/packages/ad/d4/bb94a3e3c4eeb2b97ea4ac42f5dfa71ee239150bea50c49602b2e6bb3e44/onnxsim-0.4.5.tar.gz#sha256=d691ffe81f0e1b1b4f40fbb18aca7d802e8dd3cc032db128f0e78bfa5f0caa59 has inconsistent version: filename has '0.4.5', but metadata has 'unknown' Downloading onnxsim-0.4.4.tar.gz (20.0 MB) |████████████████████████████████| 20.0 MB 27.1 MB/s Preparing metadata (setup.py) ... done WARNING: Discarding https://files.pythonhosted.org/packages/02/78/cc86d6e362a6a9fd619d49f209c34e8f1763807b46a7c53f9f7f07db2b2b/onnxsim-0.4.4.tar.gz#sha256=3333784698c9e5b3d5a05b068bede95a7e4c4945c9a513d16277ff8f30e5f4be (from https://pypi.org/simple/onnxsim/) (requires-python:>=3.6). Requested onnxsim from https://files.pythonhosted.org/packages/02/78/cc86d6e362a6a9fd619d49f209c34e8f1763807b46a7c53f9f7f07db2b2b/onnxsim-0.4.4.tar.gz#sha256=3333784698c9e5b3d5a05b068bede95a7e4c4945c9a513d16277ff8f30e5f4be has inconsistent version: filename has '0.4.4', but metadata has 'unknown' Downloading onnxsim-0.4.3.tar.gz (19.8 MB) |████████████████████████████████| 19.8 MB 29.5 MB/s Preparing metadata (setup.py) ... done WARNING: Discarding https://files.pythonhosted.org/packages/12/48/2cedf980c59e7407edcb7e823780f16a91ab38187a9e6613ce89713cb3c4/onnxsim-0.4.3.tar.gz#sha256=105afc602c1bfebecdc4785b918c0b6f6e89de1ffbcc3382fa96d1edbe38e310 (from https://pypi.org/simple/onnxsim/) (requires-python:>=3.6). Requested onnxsim from https://files.pythonhosted.org/packages/12/48/2cedf980c59e7407edcb7e823780f16a91ab38187a9e6613ce89713cb3c4/onnxsim-0.4.3.tar.gz#sha256=105afc602c1bfebecdc4785b918c0b6f6e89de1ffbcc3382fa96d1edbe38e310 has inconsistent version: filename has '0.4.3', but metadata has 'unknown'

clintlombard commented 2 years ago

You can try install using pip's legacy dependency resolver

pip3 install --use-deprecated=legacy-resolver onnxsim

As there are no aarch64/arm64 wheels, pip will attempt to build the wheel. For me it fails during the build.

paleomoon commented 2 years ago

Same issue.

GiorgosBetsos commented 2 years ago

Same issue.

I think the module requires at least Python 3.7, whereas the version of Jetson with any version of JetPack up to 4.6 is 3.6. So it won't compile.

paleomoon commented 2 years ago

@GiorgosBetsos No, I'm using JetPack 5.0 with Python 3.8. Just as mentioned above, run pip3 install --use-deprecated=legacy-resolver onnxsim will attempt to build the wheel, but failed, seems CMake 3.22 or higher is required. I gave up this for the time being.

clintlombard commented 2 years ago

@paleomoon install cmake using pip to solve that

wangxudong-cq commented 2 years ago

same issue, Can not install the latest version

CNOCycle commented 2 years ago

The same issue for me. I have tried the solution provided by https://github.com/daquexian/onnx-simplifier/issues/214#issuecomment-1203845966 and https://github.com/daquexian/onnx-simplifier/issues/214#issuecomment-1206374649. Unfortunately, those solutions did not work for me.

In my opinion, metadata has 'unknown' are shown in the output multiple times since the version cannot be resolved.

I checked file setup.py and version is resolved with the following snippet https://github.com/daquexian/onnx-simplifier/blob/ed25484364c72348b2686b442e847a05806dcc80/setup.py#L41-L47

As can be seen, version is unknown if tags information cannot be retrieve with git but I'm not sure why this issue does not happen on x86 architecture. Therefore, the solution is very simple. Clone the repository and build from the source code.

The following is the script to build wheel and install the package. I successfully built docker image run on Jetson NX.

FROM nvcr.io/nvidia/l4t-pytorch:r35.1.0-pth1.13-py3 AS pybase

# upgrade pip packages
RUN pip3 install --upgrade pip setuptools wheel

# build onnx-simplifier
FROM pybase AS onnxsim
RUN cd / && \
    git clone https://github.com/daquexian/onnx-simplifier.git -b v0.4.9 --depth 1 && \
    cd onnx-simplifier && \
    sed -i 's/git@github.com:/https:\/\/github.com\//g' .gitmodules && \
    git submodule update --init --recursive -- third_party/onnx-optimizer && \
    git submodule update --init -- third_party/onnxruntime third_party/pybind11 && \
    python3 setup.py bdist_wheel && \
    find . -name *.whl -exec pip3 install {} \; && \
    cd .. && \
    rm -rf onnx-simplifier
cycychenyi commented 1 year ago

The same issue for me. I have tried the solution provided by #214 (comment) and #214 (comment). Unfortunately, those solutions did not work for me.

In my opinion, metadata has 'unknown' are shown in the output multiple times since the version cannot be resolved.

I checked file setup.py and version is resolved with the following snippet

https://github.com/daquexian/onnx-simplifier/blob/ed25484364c72348b2686b442e847a05806dcc80/setup.py#L41-L47

As can be seen, version is unknown if tags information cannot be retrieve with git but I'm not sure why this issue does not happen on x86 architecture. Therefore, the solution is very simple. Clone the repository and build from the source code.

The following is the script to build wheel and install the package. I successfully built docker image run on Jetson NX.

FROM nvcr.io/nvidia/l4t-pytorch:r35.1.0-pth1.13-py3 AS pybase

# upgrade pip packages
RUN pip3 install --upgrade pip setuptools wheel

# build onnx-simplifier
FROM pybase AS onnxsim
RUN cd / && \
    git clone https://github.com/daquexian/onnx-simplifier.git -b v0.4.9 --depth 1 && \
    cd onnx-simplifier && \
    sed -i 's/git@github.com:/https:\/\/github.com\//g' .gitmodules && \
    git submodule update --init --recursive -- third_party/onnx-optimizer && \
    git submodule update --init -- third_party/onnxruntime third_party/pybind11 && \
    python3 setup.py bdist_wheel && \
    find . -name *.whl -exec pip3 install {} \; && \
    cd .. && \
    rm -rf onnx-simplifier

Thanks to @CNOCycle , the script used to work for me. And today I found out that the bug has been fixed in 0.4.11 with the following snippets.

https://github.com/daquexian/onnx-simplifier/blob/03c3d2f0a01e758259e19c1304864c4966d0b290/setup.py#L41-L48

https://github.com/daquexian/onnx-simplifier/blob/03c3d2f0a01e758259e19c1304864c4966d0b290/VERSION#L1

But I don’t know why, PyPi doesn‘t have v0.4.11 and v0.4.12, pip3 install onnxsim==0.4.13 works for me.