[BUG] onnxruntime automatically installed in wrong location when importing simplify

[nickfraser]

Love this tool, but it has some weird behaviour on the first import that I think should be addressed.

The Issue

Seemingly, this was done on purpose, but I cannot understand why, so I'm bringing it to your attention. Installing onnxsim (version 0.4.7) with pip doesn't install onnxruntime as a prerequisite*. Instead, it checks / installs all other dependencies and installs onnxruntime the first time you run from onnxsim import simplify.

The problem is, if you're using a virtual environment (such as conda) it will install onnxruntime in ~/.local/ instead of your chosen prefix. If that's the case, the following two things may occur:

1. Since your virtual environment doesn't search your local user prefix (~/.local) for packages, the import fails anyway**
2. If the python version you're using is different from your system python, you'll not be able to uninstall onnxruntime and it's dependencies using pip

Unless I'm missing something, I strongly recommend you report an error if onnxruntime cannot be imported rather than trying to install it automatically. I also recommend that you include onnxruntime as a prerequisite for onnxsim so that it is installed by pip when onnxsim is installed, so it is installed into the correct prefix.

Steps to Reproduce the Issue

If you have something like miniconda installed (I happen to be using miniforge):

conda create -n onnxsim_test python=3.7 # Chose something different to your system python
conda activate onnxsim_test
pip install onnxsim
python -c "from onnxsim import simplify"

Logs

• installing onnxsim. Note that onnxruntime is not installed:

$ pip install onnxsim                                                                                                                                                                                      
Collecting onnxsim
  Downloading onnxsim-0.4.7-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.0 MB)                                                                                                                                                 
     ???????????????????????????????????????? 2.0/2.0 MB 7.6 MB/s eta 0:00:00
Collecting rich                                                                                                                                                                                                                                
  Downloading rich-12.5.1-py3-none-any.whl (235 kB)
     ???????????????????????????????????????? 235.6/235.6 kB 1.3 MB/s eta 0:00:00                                                                                                                                                              
Collecting onnx
  Downloading onnx-1.12.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (13.1 MB)                                                                                                                                                  
     ???????????????????????????????????????? 13.1/13.1 MB 9.7 MB/s eta 0:00:00
Collecting numpy>=1.16.6                                                                                                                                                                                                                       
  Downloading numpy-1.21.6-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (15.7 MB)
     ???????????????????????????????????????? 15.7/15.7 MB 11.7 MB/s eta 0:00:00                                                                                                                                                               
Collecting typing-extensions>=3.6.2.1
  Downloading typing_extensions-4.3.0-py3-none-any.whl (25 kB)                                                                                                                                                                                 
Collecting protobuf<=3.20.1,>=3.12.2
  Downloading protobuf-3.20.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.0 MB)                                                                                                                                                   
     ???????????????????????????????????????? 1.0/1.0 MB 3.7 MB/s eta 0:00:00
Collecting pygments<3.0.0,>=2.6.0                                                                                                                                                                                                              
  Downloading Pygments-2.13.0-py3-none-any.whl (1.1 MB)
     ???????????????????????????????????????? 1.1/1.1 MB 8.3 MB/s eta 0:00:00                                                                                                                                                                  
Collecting commonmark<0.10.0,>=0.9.0
  Downloading commonmark-0.9.1-py2.py3-none-any.whl (51 kB)                                                                                                                                                                                    
     ???????????????????????????????????????? 51.1/51.1 kB 899.5 kB/s eta 0:00:00
Installing collected packages: commonmark, typing-extensions, pygments, protobuf, numpy, rich, onnx, onnxsim                                                                                                                                   
Successfully installed commonmark-0.9.1 numpy-1.21.6 onnx-1.12.0 onnxsim-0.4.7 protobuf-3.20.1 pygments-2.13.0 rich-12.5.1 typing-extensions-4.3.0

** Importing simplify. onnxruntime is installed in the user prefix, but fails anyway:

$ python -c "from onnxsim import simplify"                                                                                                                                                                 
Installing onnxruntime by `/proj/xlabs/users/nfraser/opt/miniforge3/envs/onnxsim_test/bin/python -m pip install --user onnxruntime`, please wait for a moment..                                                                                
Collecting onnxruntime
  Downloading onnxruntime-1.12.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.5 MB)                                                                                                                                            
     ???????????????????????????????????????? 5.5/5.5 MB 6.8 MB/s eta 0:00:00
Requirement already satisfied: numpy>=1.21.0 in /proj/xlabs/users/nfraser/opt/miniforge3/envs/onnxsim_test/lib/python3.7/site-packages (from onnxruntime) (1.21.6)                                                                             
Requirement already satisfied: protobuf in /proj/xlabs/users/nfraser/opt/miniforge3/envs/onnxsim_test/lib/python3.7/site-packages (from onnxruntime) (3.20.1)
Collecting flatbuffers                                                                                                                                                                                                                         
  Downloading flatbuffers-2.0.7-py2.py3-none-any.whl (26 kB)
Collecting coloredlogs                                                                                                                                                                                                                         
  Using cached coloredlogs-15.0.1-py2.py3-none-any.whl (46 kB)
Collecting sympy                                                                                                                                                                                                                               
  Downloading sympy-1.10.1-py3-none-any.whl (6.4 MB)
     ???????????????????????????????????????? 6.4/6.4 MB 11.7 MB/s eta 0:00:00                                                                                                                                                                 
Collecting packaging
  Using cached packaging-21.3-py3-none-any.whl (40 kB)                                                                                                                                                                                         
Collecting humanfriendly>=9.1
  Using cached humanfriendly-10.0-py2.py3-none-any.whl (86 kB)                                                                                                                                                                                 
Collecting pyparsing!=3.0.5,>=2.0.2
  Downloading pyparsing-3.0.9-py3-none-any.whl (98 kB)                                                                                                                                                                                         
     ???????????????????????????????????????? 98.3/98.3 kB 895.7 kB/s eta 0:00:00
Collecting mpmath>=0.19                                                                                                                                                                                                                        
  Using cached mpmath-1.2.1-py3-none-any.whl (532 kB)
Installing collected packages: mpmath, flatbuffers, sympy, pyparsing, humanfriendly, packaging, coloredlogs, onnxruntime                                                                                                                       
Successfully installed coloredlogs-15.0.1 flatbuffers-2.0.7 humanfriendly-10.0 mpmath-1.2.1 onnxruntime-1.12.1 packaging-21.3 pyparsing-3.0.9 sympy-1.10.1
Traceback (most recent call last):
  File "/proj/xlabs/users/nfraser/opt/miniforge3/envs/onnxsim_test/lib/python3.7/site-packages/onnxsim/onnx_simplifier.py", line 18, in <module>
    import onnxruntime as rt  # type: ignore
ModuleNotFoundError: No module named 'onnxruntime'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/proj/xlabs/users/nfraser/opt/miniforge3/envs/onnxsim_test/lib/python3.7/site-packages/onnxsim/__init__.py", line 1, in <module>
    from onnxsim.onnx_simplifier import simplify, main
  File "/proj/xlabs/users/nfraser/opt/miniforge3/envs/onnxsim_test/lib/python3.7/site-packages/onnxsim/onnx_simplifier.py", line 24, in <module>
    import onnxruntime as rt
ModuleNotFoundError: No module named 'onnxruntime'

[daquexian]

Thanks for your detailed report!

Yes, it is on purpose. It is a hard decision. The reason is onnxruntime has at least three packages on PyPI -- onnxruntime, onnxruntime-gpu, and onnxruntime-noopenmp. The ideal behavior is to install onnxruntime only if none of these packages is installed. It is not supported by pip, so I have to install onnxruntime in the runtime because then I can check if any of these packages is installed by try-catch import onnxruntime.

Since your virtual environment doesn't search your local user prefix (~/.local) for packages, the import fails anyway**

It is a little strange. onnxsim invokes pip by sys.executable -m pip so that the virtual environment should be respected. However, I admit the current way is tricky and perhaps cannot work well in some corner cases. I'm willing to change it if there is a better way.

I strongly recommend you report an error if onnxruntime cannot be imported rather than trying to install it automatically.

How about reporting an more explicit and clear error if onnxruntime cannot be imported even after automatical installation? Feel free to share your idea :)

[nickfraser]

Apologies for the delay, I missed the notification that you responded to this.

The reason is onnxruntime has at least three packages on PyPI -- onnxruntime, onnxruntime-gpu, and onnxruntime-noopenmp.

I see, thanks for this information. I think it's reasonable not to include onnxruntime in the pip dependencies for those reasons.

onnxsim invokes pip by sys.executable -m pip so that the virtual environment should be respected.

I think you are incorrect on this. You're also adding the --user flag which doesn't have clearly defined behaviour within virtual environments. See this comment. This answer also warns of the same behaviour.

Maybe an alternative solution is to simply update your installation documentation. Stating that a version of onnxruntime is a prerequisite. For example, the python installation instructions could be changed to read as follows:

pip3 install -U pip
pip3 install <onnxruntime|onnxruntime-gpu|onnxruntime-noopenmp> # Install the onnxruntime of your choice.
pip3 install onnxsim

instead of:

pip3 install -U pip && pip3 install onnxsim

How about reporting an more explicit and clear error if onnxruntime cannot be imported even after automatical installation? Feel free to share your idea :)

I still strongly recommend you remove the automatic installation. Just print an error saying:

No onnxruntime found. Please install onnxruntime. This can be done as follows:
pip3 install <onnxruntime|onnxruntime-gpu|onnxruntime-noopenmp>

This way a user can decide whether or not they use a --user flag or not. I don't think there is a clean way to do what you want to do without creating a bigger mess under some other configurations.

[CILT]

With conda maybe you want to update your library path?

You can locate the installation with:

$ pip show [some package]
...
Location: /path/to/conda/anaconda3/envs/[env]/lib/...
...

Then update the library path with: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/conda/anaconda3/envs/[env]/lib