search

maggiez0138 / Swin-Transformer-TensorRT

This project aims to explore the deployment of Swin-Transformer based on TensorRT, including the test results of FP16 and INT8.
MIT License
161 stars 29 forks source link

Does it support dynamic batch inference? #2

Closed wangjingg closed 1 year ago

wangjingg commented 2 years ago

When I try onnx->tensorrt and use dynamic batch, the tensorrt output is all [0,0,0,0,.....]

maggiez0138 commented 2 years ago

When I try onnx->tensorrt and use dynamic batch, the tensorrt output is all [0,0,0,0,.....]

Tried dynamic batch, failed. Both onnx model and tensorRT engine can not output the expected output. Maybe this issue has some relationship with the implementation of Swin Transformer. Not sure.

wangjingg commented 2 years ago

When I cancel the softmax, in pytorch->onnx(dynamic batch), It's work, you can try it,

maggiez0138 commented 2 years ago

When I cancel the softmax, in pytorch->onnx(dynamic batch), It's work, you can try it,

Thanks for your info, will do the attempt.

fatemebafghi commented 2 years ago

Hello there. I have the exact same problem with dynamic inputs, but I could not find a way to cancell softmax during converting pytorch model to onnx. Can you help me through this?

maggiez0138 commented 1 year ago

Hello there. I have the exact same problem with dynamic inputs, but I could not find a way to cancell softmax during converting pytorch model to onnx. Can you help me through this?

Updated the repo. Now dynamic input is supported.