search

neo-ai / neo-ai-dlr

Neo-AI-DLR is a common runtime for machine learning models compiled by AWS SageMaker Neo, TVM, or TreeLite.
Apache License 2.0
492 stars 106 forks source link

Not working with jetpack 4.6 #450

Closed blackrez closed 1 year ago

blackrez commented 1 year ago

Hello,

I try to load a model with dlr (with GPU enable) and I have this error : 

  File "/usr/local/lib/python3.6/dist-packages/dlr-1.10.0-py3.6.egg/dlr/api.py", line 89, in __init__
    self._impl = DLRModelImpl(model_path, dev_type, dev_id, error_log_file, use_default_dlr)
  File "/usr/local/lib/python3.6/dist-packages/dlr-1.10.0-py3.6.egg/dlr/dlr_model.py", line 78, in __init__
    self._init_libdlr()
  File "/usr/local/lib/python3.6/dist-packages/dlr-1.10.0-py3.6.egg/dlr/dlr_model.py", line 132, in _init_libdlr
    self._lib = _load_lib(find_lib_path(self.model_path, self.use_default_dlr, self.logger))
  File "/usr/local/lib/python3.6/dist-packages/dlr-1.10.0-py3.6.egg/dlr/dlr_model.py", line 39, in _load_lib
    'Error message(s): {}\n'.format(e))
dlr.dlr_model.DLRError: DLR library (libdlr.so) could not be loaded.
Likely causes:
  * OpenMP runtime is not installed (vcomp140.dll or libgomp-1.dll for Windows, libgomp.so for UNIX-like OSes)
  * You are running 32-bit Python on a 64-bit OS
Error message(s): libnvinfer.so.7: cannot open shared object file: No such file or directory

I use jetpack 4.6 on a Jetson Nano.

nabil@devkit-jetson-nano:/usr/local/lib/python3.6/dist-packages/dlr-1.10.0-py3.6.egg/dlr$ ldd libdlr.so
    linux-vdso.so.1 (0x0000007fa39f0000)
    libcudart.so.10.2 => /usr/local/cuda/lib64/libcudart.so.10.2 (0x0000007fa2c21000)
    libcuda.so.1 => /usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1 (0x0000007fa1c91000)
    libnvinfer.so.8 => /usr/lib/aarch64-linux-gnu/libnvinfer.so.8 (0x0000007f97d0f000)
    libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000007f97ce3000)
    libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x0000007f97cce000)
    libstdc++.so.6 => /usr/lib/aarch64-linux-gnu/libstdc++.so.6 (0x0000007f97b3a000)
    libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000007f97a81000)
    libgcc_s.so.1 => /lib/aarch64-linux-gnu/libgcc_s.so.1 (0x0000007f97a5d000)
    libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000007f97904000)
    /lib/ld-linux-aarch64.so.1 (0x0000007fa39c4000)
    librt.so.1 => /lib/aarch64-linux-gnu/librt.so.1 (0x0000007f978ed000)
    libnvrm_gpu.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm_gpu.so (0x0000007f978a7000)
    libnvrm.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm.so (0x0000007f97872000)
    libnvrm_graphics.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm_graphics.so (0x0000007f97852000)
    libnvidia-fatbinaryloader.so.440.18 => /usr/lib/aarch64-linux-gnu/tegra/libnvidia-fatbinaryloader.so.440.18 (0x0000007f977e1000)
    libnvdla_compiler.so => /usr/lib/aarch64-linux-gnu/tegra/libnvdla_compiler.so (0x0000007f97335000)
    libEGL.so.1 => /usr/lib/aarch64-linux-gnu/libEGL.so.1 (0x0000007f97314000)
    libnvmedia.so => /usr/lib/aarch64-linux-gnu/tegra/libnvmedia.so (0x0000007f972a7000)
    libnvos.so => /usr/lib/aarch64-linux-gnu/tegra/libnvos.so (0x0000007f97288000)
    libGLdispatch.so.0 => /usr/lib/aarch64-linux-gnu/libGLdispatch.so.0 (0x0000007f9715c000)
    libnvdc.so => /usr/lib/aarch64-linux-gnu/tegra/libnvdc.so (0x0000007f9713e000)
    libnvtvmr.so => /usr/lib/aarch64-linux-gnu/tegra/libnvtvmr.so (0x0000007f970ad000)
    libnvparser.so => /usr/lib/aarch64-linux-gnu/tegra/libnvparser.so (0x0000007f9706f000)
    libnvdla_runtime.so => /usr/lib/aarch64-linux-gnu/tegra/libnvdla_runtime.so (0x0000007f96fc8000)
    libnvimp.so => /usr/lib/aarch64-linux-gnu/tegra/libnvimp.so (0x0000007f96fb3000)
blackrez commented 1 year ago

Hello, I found the issue. It is the build from sagemaker neo. It had its own libdlr.so.

nabil@devkit-jetson-nano:~/model_gpu$ ls
code.ro  compiled.meta  compiled.so  dlr.h  libdlr.so  manifest
nabil@devkit-jetson-nano:~/model_gpu$ ldd libdlr.so
    linux-vdso.so.1 (0x0000007f91d01000)
    libcudart.so.10.2 => /usr/local/cuda/lib64/libcudart.so.10.2 (0x0000007f90f35000)
    libcuda.so.1 => /usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1 (0x0000007f8ffa5000)
    libnvinfer.so.7 => not found
    libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000007f8ff79000)
    libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x0000007f8ff64000)
    libstdc++.so.6 => /usr/lib/aarch64-linux-gnu/libstdc++.so.6 (0x0000007f8fdd0000)
    libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000007f8fd17000)
    libgcc_s.so.1 => /lib/aarch64-linux-gnu/libgcc_s.so.1 (0x0000007f8fcf3000)
    libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000007f8fb9a000)
    /lib/ld-linux-aarch64.so.1 (0x0000007f91cd5000)
    librt.so.1 => /lib/aarch64-linux-gnu/librt.so.1 (0x0000007f8fb83000)
    libnvrm_gpu.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm_gpu.so (0x0000007f8fb3d000)
    libnvrm.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm.so (0x0000007f8fb08000)
    libnvrm_graphics.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm_graphics.so (0x0000007f8fae8000)
    libnvidia-fatbinaryloader.so.440.18 => /usr/lib/aarch64-linux-gnu/tegra/libnvidia-fatbinaryloader.so.440.18 (0x0000007f8fa77000)
    libnvos.so => /usr/lib/aarch64-linux-gnu/tegra/libnvos.so (0x0000007f8fa58000)

I removed it and the it works better.