No effect on Tensorrt7

[TimVerion]

The model obtained by LFFD training is converted to ONNX. Why can it be used in Tensorrt5 but not in Tensorrt7?

A new Engine has been generated here

[YonghaoHe]

@TimVerion Of course you should rebuild the engine once the TRT version changes.

[TimVerion]

Thank you very much for your reply, but I have cleaned up the engine. Currently, the verification is carried out in c++11 tensorrt-7.0.0.11 environment. The same code can be normally reasoned on tensorrt5, but when it is changed to tensorrt7, the feature diagram will not light up

[YonghaoHe]

@TimVerion Maybe you should take a look at the version compatibility between MXNET, ONNX used in MXNET, TensorRT and ONNX used in TensorRT.

[TimVerion]

Follow your advice, I'm mxnet 1.5.1 and 1.6.0 onnx 1.2.1 to 1.4.1 tried respectively, then also tried on the python code of reasoning, generate the engine without error, but there is no light at the time of reasoning characteristic figure, so if you have tried tensorrt7.0 and appear as a result, can put your environment version number told it to me? Thank you very much!!

[GilbertTam]

I has the same question @TimVerion, Has any suggestion to solve this problem?

[TimVerion]

Try to match various versions, but it still has no effect.

[YonghaoHe]

@TimVerion Recently, I do not work on MXNet, and switch to PyTorch. In PyTorch, I can successfully use TRT 7.0. I will make a whole new repo available as soon as possible. I think if you can ask for help in MXNet community.

[TimVerion]

Thank you for your reply, and I am looking forward to the release of your Pytorch version.

[103061634]

@YonghaoHe Hi, can you share about switch MXNet to PyTorch. THX!!

[YonghaoHe]

@103061634 We are still working on PyTorch version which will be epanded to multi-class detection with some new features. This version will be released in the folowing two months.

[HT-Yuan]

Thank you for your reply, and I am looking forward to the release of your Pytorch version.

Hi, I also encountered this problem and found that it was caused by the onnx version, but I don’t know the original onnx version number of the project. Have you solved this problem? Can you provide some suggestions? (Or some other face detection solutions that are quickly used in tensorRT7)

[royinx]

I dig into and found it is caused by the softmax alignment between TensorRT and ONNXruntime.

For TensorRT <=6 is like mxnet , softmax by axis, For TensorRT 7, it follows onnxruntime, that will flatten the tensor into 2D array before exponential if shape > 2. reference so TRT 7 will not support softmax axis > 1 ( dimension > 2 )

---

TRT7 assume the first dimension is N and flatten the remaining dimension. e.g (3,4,5,6) -> (3,120)

so I transpose the axis as the last layer, and reshape into 2D, e.g for axis = 1 , (3,4,5,6) -> (3,5,6,4) -> (90,4) for axis = 2 , (3,4,5,6) -> (3,4,6,5) -> (72,5) for axis = 3 , (3,4,5,6) -> (3,4,5,6) -> (60,6)

---

so I modify the below code to transpose > reshape > softmax > transpose > reshape

change this line in symbol_farm to

layer_shape = {"conv8" :( -1, 119, 159 ,2),
               "conv11":( -1, 59, 79 ,2),
               "conv14":( -1, 29, 39 ,2),
               "conv17":( -1, 14, 19 ,2),
               "conv20":( -1, 6, 9 ,2)}

if deploy_flag:
    predict_score = mxnet.symbol.transpose(data=branch_conv3_score, axes=(0,2,3,1))
    predict_score = mxnet.symbol.reshape(data=predict_score, shape=(-1,2))
    predict_score = mxnet.symbol.softmax(data=predict_score, axis=1)
    predict_score = mxnet.symbol.reshape(data=predict_score, shape=layer_shape[prefix_name])
    predict_score = mxnet.symbol.transpose(data=predict_score, axes=(0,3,1,2))
    predict_score = mxnet.symbol.slice_axis(data=predict_score, axis=1, begin=0, end=1)

onnx version: onnx==1.4

python3 symbol_farm/symbol_10_320_20L_5scales_v2.py
python3 deploy_tensorrt/to_onnx.py

and compile the engine with

trtexec --onnx=trt7.onnx --saveEngine=trt7.plan --fp16 --verbose --dumpOutput --explicitBatch

for docker user, you can run this under the onnx folder

docker run --rm -it \
           --runtime nvidia \
           -v ${PWD}:/workdir \
           -w /workdir \
           nvcr.io/nvidia/tensorrt:21.03-py3 \
           trtexec --onnx=trt7.onnx --saveEngine=trt7.plan --fp16 --verbose --dumpOutput --explicitBatch