incorrect output shape after loading onnx model under c++

[ystsaan]

Hi there, I built three networks from onnx files under c++ by referring https://github.com/NVIDIA/TensorRT/blob/master/samples/opensource/sampleOnnxMNIST/sampleOnnxMNIST.cpp and checked the network output shape by modifying the following script in bool SampleOnnxMNIST::build() function from line 144 to 150

assert(network->getNbInputs() == 1);
mInputDims = network->getInput(0)->getDimensions();
assert(mInputDims.nbDims == 4);

assert(network->getNbOutputs() == 1);
mOutputDims = network->getOutput(0)->getDimensions();
assert(mOutputDims.nbDims == 2);

into

// network input shape check
assert(network->getNbInputs() == 1);
mInputDims = network->getInput(0)->getDimensions();
std::cout << (mInputDims.nbDims) << std::endl;
for (int i = 0; i < mInputDims.nbDims; i++) {
if (i != (mInputDims.nbDims-1)) {
std::cout << mInputDims.d[i] << " ";
}
else {
std::cout << mInputDims.d[i] << std::endl;
}

// network output shape check
}
assert(network->getNbOutputs() == 1);
mOutputDims = network->getOutput(0)->getDimensions();
std::cout << (mOutputDims.nbDims) << std::endl;
for (int i = 0; i < mOutputDims.nbDims; i++) {
if (i != (mOutputDims.nbDims - 1)) {
std::cout << mOutputDims.d[i] << " ";
}
else {
std::cout << mOutputDims.d[i] << std::endl;
}
}

Here is the c++ console output.

[04/30/2021-14:19:35] [I] Building and running a GPU inference engine for Cardiac Segmentation
----------------------------------------------------------------
Input filename:   C:\Users\shaorenjie\Desktop\\deeplab_v3_echonet_dynamic_mse.onnx
ONNX IR version:  0.0.6
Opset version:    12
Producer name:    pytorch
Producer version: 1.8
Domain:
Model version:    0
Doc string:
----------------------------------------------------------------
[04/30/2021-14:19:36] [I] [TRT] Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
[04/30/2021-14:19:56] [I] [TRT] Detected 1 inputs and 1 output network tensors.
1 3 112 112     
-1 -1 -1 -1       

The last two lines of console output are network input shape and output shape where output shape should be 1 2 112 112 but output -1 -1 -1 -1. So what is the problem here? I also tried other onnx files with Opset version: 10 and the console outputs are correct. Does the problem occur due to opset version or just the onnx model contains some operators that tensorrt does not support?

[ttyio]

Hello @ystsaan , could you take a try with the get_shape in the IExecutionContext which will involve shape inference? thanks https://docs.nvidia.com/deeplearning/tensorrt/api/python_api/infer/Core/ExecutionContext.html#tensorrt.IExecutionContext.get_shape

[ystsaan]

the problem is caused by the output of inbuilt pytorch segmentation neural network which is a dict and does not have shape attribute. "model(input)['out']" is required to get the final output the neural network. The output of onnx file passed to TensorRT should not be a dict. Thanks very much.