search

nnstreamer / nnstreamer

:twisted_rightwards_arrows: Neural Network (NN) Streamer, Stream Processing Paradigm for Neural Network Apps/Devices.
https://nnstreamer.ai
GNU Lesser General Public License v2.1
683 stars 171 forks source link

Does NNstream support Jetson board such as Jetson AGX Orin, and ML framwork TensorRT? #4459

Open liuhao-97 opened 1 month ago

liuhao-97 commented 1 month ago

Does NNstream support Jetson board such as Jetson AGX Orin, and ML framwork TensorRT? Besides, is it possible to use fit nnstream into NVIDIA deepstream?

Thanks!

taos-ci commented 1 month ago

:octocat: cibot: Thank you for posting issue #4459. The person in charge will reply soon.

myungjoo commented 1 month ago
  1. Yes, TensorRT is supported.
  2. Yes, as long as you implement a decoder/converter between deepstream streams and nnstreamer streams.
    • Deepstream has "tensors" as metadata of media streams; not considering tensors as native data types of gstreamer.
    • NNStreamer has "tensors" as yet another media streams; considering tensors as native data types.
liuhao-97 commented 1 month ago

Thanks for answering. @myungjoo

BTW does NNStreamer accept Udp package with raw tensor as udp input? I am asking because I am doing model splitting, which the model is splitted into two parts and these two parts are executing on two devices, and the intermediate feature map are sent from one device to the other. I am wondering if NNstreamer support this feature.

Thanks!

myungjoo commented 1 month ago

BTW does NNStreamer accept Udp package with raw tensor as udp input?

Yes. Actually, that's what GStreamer's base plugins do, udpsrc and udpsink, if what you want is a raw & basic connectivity. https://gstreamer.freedesktop.org/documentation/udp/udpsrc.html?gi-language=c

For example. if you are accepting UDP packets of tensors:

udpsrc port=10000 ! other/tensors,format=static,dims=.... ! tensor_filter .... ! ....

If you want something more than that, you may use edgesrc/edgesink with TCP or MQTT/Hybrid.