search

dusty-nv / jetson-inference

Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson.
https://developer.nvidia.com/embedded/twodaystoademo
MIT License
7.78k stars 2.98k forks source link

HOW to convert .onnx file to tensorRT engine file using [jetson-inference]? #997

Closed leaf918 closed 3 years ago

leaf918 commented 3 years ago

the engine file may be used in the deepstream applications.

Thanks.

dusty-nv commented 3 years ago

Hmm, I'm not sure of the particular requirements of the TensorRT engine needed for DeepStream, but you could just run imagenet/detectnet/segnet with your model, and it will create the .engine file for you during the loading process. That .engine file gets saved to the directory that the original model was located.

Or you could use the trtexec tool (found under /usr/src/tensorrt/bin). You can instruct that tool to save the engine for you from ONNX.

priyaganaboor-hash commented 3 years ago

Hey @leaf918,

Have you found the solution?? I have converted pytorch to onnx in my local PC. Now I want convert onnx to tensorrt on jetson nano, the @dusty-nv script worked for me. But I wanted to write code to convert it. Can you please help??

dusty-nv commented 3 years ago

@priyaganaboor-hash are you trying to run the ONNX model through jetson-inference library? If so, just copy the ONNX model to Jetson and try to run it through imagenet/detectnet/ect. It will automatically create TensorRT engine. The TensorRT engine needs to be created on the type of GPU that you want to run it on, so you can't create TensorRT engine on a PC and run it on a Jetson. You also can't copy TensorRT engine between Nano and Xavier for example. But you can copy TensorRT engine between Nano's (because they use same GPU)

If you want to make TensorRT engine for a different purpose, the trtexec tool can save the engine for you. It is found under /usr/src/tensorrt/bin (on Jetson)

priyaganaboor commented 3 years ago

Thanks @dusty-nv ,

Actually, I used the imanet.py and converted my .onnx to TensorRT and tested the working using the live camera as well as individual pictures. I understood your code on the training model, testing, and even converting it to .onnx. After this, I didn't understand how are you generating the engine file from onnx file, and processing the images from the camera to classify.

This topic is still confusing. Any information regarding this will be helpful.

leaf918 commented 3 years ago

Hi I have find the best solution for jetson tensortRT inference scenario.

1.https://github.com/dusty-nv/jetson-inference is a very important and great repo and reference stuff for jetson-like tensorrt runtime.Such like memory control,trt plugin.But the destination of this repo is demonstrating the jetson ability ,especially on detect/classification.But it's not for user special destination,for example,OCR.

2.The best converting tool which can convert pytorch inference model to tensorRT mode(engine file or plan file) is Torch2trt,It will use any onnx model ,it's just use the pytorch model's weights and tensorrt API.any will build the corresponding tensorrt Network object,and it can be save ,build ,then be executed.

3.pytorch -training->torch2trt - save engine file -> deepstream scenario(or jetson-inference repo) .

Thanks.

@rpatton-nv @dusty-nv