NVIDIA / TensorRT

NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
https://developer.nvidia.com/tensorrt
Apache License 2.0
10.15k stars 2.08k forks source link

TensorRT 10.1 and TensorRT 8.6 api update #3954

Closed hch-baobei closed 1 week ago

hch-baobei commented 2 weeks ago

tensorrt.ICudaEngine.get_binding_index(tensor name:str)

The above is the usage before version 8.6. I want to know which API can be used in 10.1 to achieve the same effect. The warning in 8.6 suggests get_tensor_name, but after checking the API documentation, I found that it cannot be equivalently replaced.

Then I found that direct [] can be equivalent in 8.6, but an error is reported in 10.1.

so which API equivalent should I use in 10.1?

lix19937 commented 1 week ago

No name-based equivalent replacement, you should use multi apis set.

hch-baobei commented 1 week ago

I checked the API documentation and it seems that I can only get a dict of index and name through get_tensor_name. Which APIs do you think can be used to achieve this?

lix19937 commented 1 week ago

For you ref

    for i in range(engine.num_io_tensors):
        tensor_name = engine.get_tensor_name(i)
        size = trt.volume(engine.get_tensor_shape(tensor_name))
        dtype = trt.nptype(engine.get_tensor_dtype(tensor_name))

        ...
       # Append to the appropriate input/output list.
        if engine.get_tensor_mode(tensor_name) == trt.TensorIOMode.INPUT:
            ...
        else:
            ...