Closed seungtaek94 closed 2 years ago
Please provide the complete log of "onnx_to_tensorrt.py".
@jkjung-avt
(trt) seungtaek@seungtaek:~/trt/tensorrt_demos/yolo$ python onnx_to_tensorrt.py -m yolov4-p5-640x384
Loading the ONNX file...
Adding yolo_layer plugins.
Traceback (most recent call last):
File "onnx_to_tensorrt.py", line 212, in <module>
main()
File "onnx_to_tensorrt.py", line 201, in main
args.model, args.int8, args.dla_core, args.verbose)
File "onnx_to_tensorrt.py", line 123, in build_engine
network = add_yolo_plugins(network, model_name, TRT_LOGGER)
File "/home/seungtaek/trt/tensorrt_demos/yolo/plugins.py", line 123, in add_yolo_plugins
raise RuntimeError('cannot get YoloLayer_TRT plugin creator')
RuntimeError: cannot get YoloLayer_TRT plugin creator
Here :)
You are not using the latest code... Please git pull
the latest code, rebuild the "yolo_layer" plugin (make clean; make
), and re-run "yolo_to_onnx.py" and "onnx_to_tensorrt.py".
The raise RuntimeError(...)
is on line 122 (not 123) in the latest code.
@jkjung-avt
def get_plugin_creator(plugin_name, logger):
"""Get the TensorRT plugin creator."""
trt.init_libnvinfer_plugins(logger, '')
plugin_creator_list = trt.get_plugin_registry().plugin_creator_list
for c in plugin_creator_list:
#print(c.name) # << here
if c.name == plugin_name:
return c
return None
No, Im using latest version. I added only one line that print(c.name)
in plugins.py
. So that is why line number is different with latest code.
If this ctypes.cdll.LoadLibrary('../plugins/libyolo_layer.so')
succeeds, "YoloLayer_TRT" plugin creator should have been registered into the TensorRT...
Do you see something similar when you do print(c.name)
on all TensorRT plugin creators?
Otherwise, which version of TensorRT are you using?
@jkjung-avt
I use TensorRT-7.1.3.4
with cuda 11.1.
And below is print(c.name)
output.. YoloLayer_TRT
is not registered.. But i dont know why.
CustomQKVToContextPluginDynamic
CustomEmbLayerNormPluginDynamic
RnRes2Br1Br2c_TRT
RnRes2Br1Br2c_TRT
CustomSkipLayerNormPluginDynamic
CustomSkipLayerNormPluginDynamic
CustomSkipLayerNormPluginDynamic
GroupNormalizationPlugin
CustomGeluPluginDynamic
CustomEmbLayerNormPluginDynamic
CgPersistentLSTMPlugin_TRT
CustomQKVToContextPluginDynamic
CustomQKVToContextPluginDynamic
CustomFCPluginDynamic
SingleStepLSTMPlugin
RnRes2Br2bBr2c_TRT
RnRes2Br2bBr2c_TRT
RnRes2FullFusion_TRT
SmallTileGEMM_TRT
RNNTEncoderPlugin
GridAnchor_TRT
NMS_TRT
Reorg_TRT
Region_TRT
Clip_TRT
LReLU_TRT
PriorBox_TRT
Normalize_TRT
RPROI_TRT
BatchedNMS_TRT
BatchedNMSDynamic_TRT
FlattenConcat_TRT
CropAndResize
DetectionLayer_TRT
Proposal
ProposalLayer_TRT
PyramidROIAlign_TRT
ResizeNearest_TRT
Split
SpecialSlice_TRT
InstanceNormalization_TRT
Beats me...
Did you see any errors/warnings when building the plugin?
@jkjung-avt
Below is all outputs when i do make
in plugins forlder.
computes: 61
NVCCFLAGS: -gencode arch=compute_61,code=[sm_61,compute_61]
nvcc -ccbin g++ -I"/usr/local/cuda-11.1/include" -I"/usr/local/TensorRT-7.1.3.4/include" -I"/usr/local/include" -I"plugin" -gencode arch=compute_61,code=[sm_61,compute_61] -Xcompiler -fPIC -c -o yolo_layer.o yolo_layer.cu
yolo_layer.h(89): warning: function "nvinfer1::IPluginV2Ext::configurePlugin(const nvinfer1::Dims *, int32_t, const nvinfer1::Dims *, int32_t, const nvinfer1::DataType *, const nvinfer1::DataType *, const __nv_bool *, const __nv_bool *, nvinfer1::PluginFormat, int32_t)" is hidden by "nvinfer1::YoloLayerPlugin::configurePlugin" -- virtual function override intended?
yolo_layer.h(89): warning: function "nvinfer1::IPluginV2Ext::configurePlugin(const nvinfer1::Dims *, int32_t, const nvinfer1::Dims *, int32_t, const nvinfer1::DataType *, const nvinfer1::DataType *, const bool *, const bool *, nvinfer1::PluginFormat, int32_t)" is hidden by "nvinfer1::YoloLayerPlugin::configurePlugin" -- virtual function override intended?
g++ -shared -o libyolo_layer.so yolo_layer.o -L"/usr/local/cuda-11.1/lib64" -L"/usr/local/TensorRT-7.1.3.4/lib" -L"/usr/local/lib" -Wl,--start-group -lnvinfer -lnvparsers -lnvinfer_plugin -lcudnn -lcublas -lnvToolsExt -lcudart -lrt -ldl -lpthread -Wl,--end-group
Below is all outputs when i do make in plugins folder.
The log looks completely normal. (The virtual function override warnings could be safely ignored.) I don't really have on idea why you met this problem ("cannot get YoloLayer_TRT plugin creator")...
I'm closing this issue since I myself am not able to reproduce the problem.
Hello there,
I had exactly the same error, I could not get the script "onnx_to_tensorrt.py -m yolov4-416" to work. Has anyone found the solution?
@sebawsky Did you run "onnx_to_tensorrt.py" in the "yolo" subdirectory?
Refer to the relevant source code below.
Are you able to see the "libyolo_layer.so" file if you do ls -l ../plugins/libyolo_layer.so
?
@jkjung-avt Yes, I ran as it appears in the tutorial. I followed the tutorial on my Jetson Nano and I was able to make the network work without problems but in ubuntu18 I could not (I still have this error).
Greetings
@sebawsky Make sure your TensorRT libraries (libnvinfer.so, etc.) are in your shared library paths.
Im also getting the same issue
i have installed TensorVersion: 8.0.1.6 on tesla v4 (ubuntu 20.04 ) amazon machine
File "onnx_to_tensorrt.py", line 212, in
cannot get YoloLayer_TRT plugin creator
i have sucesfully converte3d yolo yo onnx , but getting the exactly same issue, Even i checked path of libyolo_layer.so & added here still getting the same issue
try: ctypes.cdll.LoadLibrary('/home/ubuntu/tensorrt_demos/plugins/libyolo_layer.so')
Your help is really appreciated.Thanks @jkjung-avt
i have attachd all screenshots ,please have look cuda 11.3 tensorrt 8.0.1.6 ubantu 20.04 tesla v4
also i have printed this C_NAMES_ CustomQKVToContextPluginDynamic C_NAMES___ CustomEmbLayerNormPluginDynamic C_NAMES___ RnRes2Br1Br2c_TRT CNAMES RnRes2Br1Br2c_TRT C_NAMES_ CustomSkipLayerNormPluginDynamic C_NAMES___ CustomSkipLayerNormPluginDynamic C_NAMES___ CustomSkipLayerNormPluginDynamic C_NAMES___ GroupNormalizationPlugin C_NAMES___ CustomGeluPluginDynamic C_NAMES___ CustomEmbLayerNormPluginDynamic C_NAMES___ CgPersistentLSTMPlugin_TRT CNAMES CustomQKVToContextPluginDynamic C_NAMES_ CustomQKVToContextPluginDynamic C_NAMES___ CustomFCPluginDynamic C_NAMES___ SingleStepLSTMPlugin C_NAMES___ RnRes2Br2bBr2c_TRT CNAMES RnRes2Br2bBr2c_TRT C_NAMES_ RnRes2FullFusion_TRT CNAMES SmallTileGEMM_TRT C_NAMES_ RNNTEncoderPlugin C_NAMES___ GridAnchor_TRT CNAMES NMS_TRT C_NAMES_ Reorg_TRT CNAMES Region_TRT C_NAMES_ Clip_TRT CNAMES LReLU_TRT C_NAMES_ PriorBox_TRT CNAMES Normalize_TRT C_NAMES_ RPROI_TRT CNAMES BatchedNMS_TRT C_NAMES_ BatchedNMSDynamic_TRT CNAMES FlattenConcat_TRT C_NAMES_ CropAndResize C_NAMES___ DetectionLayer_TRT CNAMES Proposal C_NAMES_ ProposalLayer_TRT CNAMES PyramidROIAlign_TRT C_NAMES_ ResizeNearest_TRT CNAMES Split C_NAMES_ SpecialSlice_TRT CNAMES InstanceNormalization_TRT
Please check the following (an example when I run it on my Jetson Xavier NX, JetPack-4.5). Make sure "ldd" is able to find all libraries which "libyolo_layer.so" is dependent upon.
$ cd yolo/
$ ldd ../plugins/libyolo_layer.so
linux-vdso.so.1 (0x0000007fac0a2000)
libnvinfer.so.7 => /usr/lib/aarch64-linux-gnu/libnvinfer.so.7 (0x0000007f9ebcc000)
libcudart.so.10.2 => /usr/local/cuda/lib64/libcudart.so.10.2 (0x0000007f9eb58000)
libstdc++.so.6 => /usr/lib/aarch64-linux-gnu/libstdc++.so.6 (0x0000007f9e9c4000)
libgcc_s.so.1 => /lib/aarch64-linux-gnu/libgcc_s.so.1 (0x0000007f9e9a0000)
libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000007f9e847000)
/lib/ld-linux-aarch64.so.1 (0x0000007fac076000)
libcublas.so.10 => /usr/lib/aarch64-linux-gnu/libcublas.so.10 (0x0000007f99b69000)
libcudnn.so.8 => /usr/lib/aarch64-linux-gnu/libcudnn.so.8 (0x0000007f99b2b000)
libmyelin.so.1 => /usr/lib/aarch64-linux-gnu/libmyelin.so.1 (0x0000007f9969d000)
libnvrtc.so.10.2 => /usr/local/cuda/lib64/libnvrtc.so.10.2 (0x0000007f9826c000)
libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x0000007f98257000)
libnvdla_compiler.so => /usr/lib/aarch64-linux-gnu/tegra/libnvdla_compiler.so (0x0000007f97e23000)
libnvmedia.so => /usr/lib/aarch64-linux-gnu/tegra/libnvmedia.so (0x0000007f97db9000)
libEGL.so.1 => /usr/lib/aarch64-linux-gnu/libEGL.so.1 (0x0000007f97d98000)
libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000007f97cdf000)
libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000007f97cb3000)
librt.so.1 => /lib/aarch64-linux-gnu/librt.so.1 (0x0000007f97c9c000)
libcublasLt.so.10 => /usr/lib/aarch64-linux-gnu/libcublasLt.so.10 (0x0000007f95cd6000)
libnvos.so => /usr/lib/aarch64-linux-gnu/tegra/libnvos.so (0x0000007f95cb8000)
libnvrm.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm.so (0x0000007f95c75000)
libnvrm_graphics.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm_graphics.so (0x0000007f95c55000)
libnvdc.so => /usr/lib/aarch64-linux-gnu/tegra/libnvdc.so (0x0000007f95c35000)
libnvtvmr.so => /usr/lib/aarch64-linux-gnu/tegra/libnvtvmr.so (0x0000007f95ba5000)
libnvparser.so => /usr/lib/aarch64-linux-gnu/tegra/libnvparser.so (0x0000007f95b69000)
libGLdispatch.so.0 => /usr/lib/aarch64-linux-gnu/libGLdispatch.so.0 (0x0000007f95a3d000)
libnvimp.so => /usr/lib/aarch64-linux-gnu/tegra/libnvimp.so (0x0000007f95a28000)
This I get:
$ cd yolo/ $ ldd ../plugins/libyolo_layer.so linux-vdso.so.1 (0x00007fff56f5f000) libnvinfer.so.7 => /usr/lib/x86_64-linux-gnu/libnvinfer.so.7 (0x00007fa0e9022000) libcudart.so.10.2 => /usr/local/cuda-10.2/lib64/libcudart.so.10.2 (0x00007fa0e8da4000) libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fa0e8a1b000) libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fa0e8803000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa0e8412000) libcudnn.so.7 => /usr/lib/x86_64-linux-gnu/libcudnn.so.7 (0x00007fa0ccb28000) libcublas.so.10 => /usr/local/cuda-10.2/lib64/libcublas.so.10 (0x00007fa0c8871000) libmyelin.so.1 => /usr/lib/x86_64-linux-gnu/libmyelin.so.1 (0x00007fa0c805f000) libnvrtc.so.10.2 => /usr/local/cuda-10.2/lib64/libnvrtc.so.10.2 (0x00007fa0c68b2000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fa0c66ae000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fa0c6310000) /lib64/ld-linux-x86-64.so.2 (0x00007fa0f7fb5000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fa0c60f1000) librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fa0c5ee9000) libcublasLt.so.10 => /usr/local/cuda-10.2/lib64/libcublasLt.so.10 (0x00007fa0c4054000)
Thank @jkjung-avt
@jkjung-avt im running this on ubantu 20.04 server
cd tensorrt_demos/yolo ldd ../plugins/libyolo_layer.so
linux-vdso.so.1 (0x00007ffefbbdb000)
libnvinfer.so.8 => /lib/x86_64-linux-gnu/libnvinfer.so.8 (0x00007f0f8e3c6000)
libcudart.so.11.0 => /usr/local/cuda/targets/x86_64-linux/lib/libcudart.so.11.0 (0x0007f0f8e12d000)
libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f0f8df4b000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f0f8df30000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f0f8dd3e000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f0f8dd31000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f0f8dd2b000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f0f8dbdc000)
/lib64/ld-linux-x86-64.so.2 (0x00007f0fa66c5000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f0f8dbb9000)
im getting only this files, ( actually last time i was able to convert all custom models( scaled yolo) into tensorrt on the same server with same setup ,but now im getting this issue) How should i resolve this issue? should i need to uninstall all tensorrt ,pycuda,onnx ? Thanks always @jkjung-avt
@sebawsky @akashAD98 It is OK as long as all .so files could be found. So the problems you've encountered are not due to missing dynamic libraries.
Could you try the following instead? Show me the exception logs, and I'll see how to further debug from there.
$ cd yolo/
$ python3 -c "import ctypes; ctypes.cdll.LoadLibrary('../plugins/libyolo_layer.so') "
@jkjung-avt my server is off , once its switched on ill do the testing
@jkjung-avt I ran the indicated line and didn't get any exceptions (the answer was clean).
@jkjung-avt same thing, there is no output, what should we do next?
@sebawsky @akashAD98 I see. So the problem is not loading of the dynamic library ("libyolo_layer.so.so"). Rather, REGISTER_TENSORRT_PLUGIN()
does not work as expected ("YoloLayer_TRT" plugin creator is not registered).
The relevant source code is here. I don't know why, but will investigate more when I have time.
@jkjung-avt is there any temporary solution for this? it's a little bit urgent for me, Should we try a new setup? new machine? any advice will be really helpful
@jkjung-avt @sebawsky I solved this issue by uninstalling the old tensorrt version.pycuda & onxx & i did again setup with TensorRT-8.0.3.4 ,& its working fine . & again thanks for your valuable time & reply ):
@jkjung-avt I have converted yolo to onnx & onnx to tensorrt, But at the time of doing testing i was not able to get detection. bcz my pycuda is installed but its showing not installed,here its log
@akashAD98 Please remove the whole "src/pycuda-2019.1.2" directory and re-run "install_pycuda.sh".
$ rm -rf ${HOME}/src/pycuda-2019.1.2/
$ cd ${HOME}/project/tensorrt_demos/ssd/
$ ./install_pycuda.sh
I had the same issue of "RuntimeError: cannot get YoloLayer_TRT plugin creator", because the environment CUDA_VISIBLE_DEVICES was set to -1. after export CUDA_VISIBLE_DEVICES=0, it worked.
@sebawsky @akashAD98 I see. So the problem is not loading of the dynamic library ("libyolo_layer.so.so"). Rather,
REGISTER_TENSORRT_PLUGIN()
does not work as expected ("YoloLayer_TRT" plugin creator is not registered).The relevant source code is here. I don't know why, but will investigate more when I have time.
@jkjung-avt , I am still confused. I can't go any further than this. Did you happen to check this?
Hi. I try to convert yolo to trt.
I succeed to convert
.cfg -> .onnx
but failed.onnx -> .trt
with Error "cannot get YoloLayer_TRT plugin creator".What should I check now and what does mean that error message.?
I already
make
in plugins dir and getlibyolo_layer.so
lib.