search

dusty-nv / jetson-inference

Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson.
https://developer.nvidia.com/embedded/twodaystoademo
MIT License
7.74k stars 2.97k forks source link

Example of TensorRT SSD and Pytorch #1382

Closed rsamvelyan closed 2 years ago

rsamvelyan commented 2 years ago

Hello

I need much more flexibility as far as the window with the webcam feed is concerned. I need to use OpenCV in order to draw my custom boxes and labels. Something that the jetson-inference doesn't allow me to do.

Where could I get examples of how to integrate OpenCV webcam with Pytorch SSD inference and TensorRT optimization?

In other words I'm looking for a reference that would allow me to re-create the same thing as jetson-inference does but in Python.

Thanks!

dusty-nv commented 2 years ago

Hi @rsamvelyan, you can use the cudaFromNumpy() and cudaToNumpy() functions from jetson.utils to convert images to/from OpenCV: https://github.com/dusty-nv/jetson-inference/blob/master/docs/aux-image.md#converting-to-numpy-arrays

rsamvelyan commented 2 years ago

Ok great! Will give it a try. I have my custom trained PTH files which work very well with jetson-inference but so far I need to convert them to ONNX. Is there a way to run inference on the .pth files directly instead of heaving to convert them to .onnx?

dusty-nv commented 2 years ago

Is there a way to run inference on the .pth files directly instead of heaving to convert them to .onnx?

No, because jetson-inference runtime library doesn't use PyTorch - it uses TensorRT, and TensorRT imports ONNX

rsamvelyan commented 2 years ago

OK I see. And if I wanted to use directly PHT files then I would have to write my own inference from scratch?

dusty-nv commented 2 years ago

And if I wanted to use directly PHT files then I would have to write my own inference from scratch?

You would just use PyTorch to run PTH checkpoints...

rsamvelyan commented 2 years ago

And in that case would not be able to optimize it with TensroRT, correct?

dusty-nv commented 2 years ago

Correct, although you can try use torch2trt which uses TensorRT and doesn't need ONNX: https://github.com/NVIDIA-AI-IOT/torch2trt

rsamvelyan commented 2 years ago

Thanks as always Dusty! Really appreciate your help. My projects would probably have stalled without your help!!

dusty-nv commented 2 years ago

No problem @rsamvelyan, happy to help! Wish you the best of luck on your project!

rsamvelyan commented 2 years ago

I created the live webcam capture that uses OPEN CV. It works very well and the FPS is very high - over 30 FPS on Jetson Nano. I can share it if you want.