search

NVIDIA / TensorRT

NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
https://developer.nvidia.com/tensorrt
Apache License 2.0
10.81k stars 2.13k forks source link

Global tensors with dynamic slice using python #4229

Open zengrh3 opened 2 weeks ago

zengrh3 commented 2 weeks ago

Let say I define a global tensor t1, where shape is [1, 12, 2000, 112], and the actual input tensor t2 shape is [1, 12, 40, 112]. I have copied the actual value of t2 to t1 using _copy, and in the inner model logic, I use slice to get the actual content, like t1[:, :, :40, :].

However, I found this does not work in TensorRT, but works in ONNX. So how can I deal with this ?

yuanyao-nv commented 2 weeks ago

Are you able to create an ONNX model with this? You can try parsing an ONNX model with TensorRT https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#onnx-intro