search

NVIDIA / TensorRT

NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
https://developer.nvidia.com/tensorrt
Apache License 2.0
10.78k stars 2.13k forks source link

My onnx file successfully run by CUDAExecutionProvider, but failed with TensorrtExecutionProvider through onnxruntime #4109

Closed d5423197 closed 2 months ago

d5423197 commented 2 months ago

I trained a tensorflow model and converted to onnx model file using tf2onnx.

This onnx model can run successfully on CUDAExecutionProvider.

But when I tried to run it on TensorrtExecutionProvider . It shows a dimension error when going through a concatenate layer.

the merge6, the error is like (2, 64, 64, 256) can not be concatenated with (0, 64, 64, 256)。I think the dimension here is wrong.

above is the model i used which is called bcdu-net.

model structure: https://github.com/rezazad68/BCDU-Net/blob/master/Lung%20Segmentation/models.py

Please help

d5423197 commented 2 months ago

2024-09-05 15:04:08.383078845 [E:onnxruntime:Default, tensorrt_execution_provider.h:58 log] [2024-09-05 07:04:08 ERROR] [concatenationLayer.cpp::estimateOutputDims::110] Error Code 4: Internal Error ((Unnamed Layer* 77) [Concatenation]: all concat input tensors must have the same dimensions except on the concatenation axis (1), but dimensions mismatched at index 0. Input 0 shape: [2,64,64,256], Input 1 shape: [0,64,64,256])

d5423197 commented 2 months ago

I have confirmed that, once my model contain tensorflow convLSTM2D layer. Tensorrt failed to build. Does tensorrt support convLSTM2D?

moraxu commented 2 months ago

Hi, could you please provide more info like the TRT version, OS version etc? I will open an internal bug then. You can use the template from "Report a TensorRT issue" when opening a new issue and paste it here.

Image

lix19937 commented 2 months ago

I have confirmed that, once my model contain tensorflow convLSTM2D layer. Tensorrt failed to build. Does tensorrt support convLSTM2D?

You can try to use trtexec.

d5423197 commented 2 months ago

@moraxu Hello there, thanks for your reply. I have created another issue. I think the info is enough to reproduce. Please let me know if any other info needed.

moraxu commented 2 months ago

Closing in favor of https://github.com/NVIDIA/TensorRT/issues/4117