Hardware:
GPU: Tesla T4
Software:
tensorrt-7.1.3
cudnn - 8
I was wondering if I already have a tensorrt engine build, how to use tkDNN to fast up my inference.
Currently we are facing an issue while using scaledyolov4, inference is linear with batching, so we want to reduce inference further.
I already have tensorrt engine built for 1280x1280 for batchsize 1,4,16. I was wondering how to use tkDNN along with it.
TIY!
Hardware: GPU: Tesla T4 Software: tensorrt-7.1.3 cudnn - 8 I was wondering if I already have a tensorrt engine build, how to use tkDNN to fast up my inference. Currently we are facing an issue while using scaledyolov4, inference is linear with batching, so we want to reduce inference further. I already have tensorrt engine built for 1280x1280 for batchsize 1,4,16. I was wondering how to use tkDNN along with it. TIY!