search

ceccocats / tkDNN

Deep neural network library and toolkit to do high performace inference on NVIDIA jetson platforms
GNU General Public License v2.0
718 stars 209 forks source link

Is there a way similar to trtexec --loadEngine for just testing the speed of the model? #202

Closed ysyyork closed 3 years ago

ysyyork commented 3 years ago

For pure tensorrt engines, I can easily test the throughput by trtexec --loadEngine and it will randomly generate the inputs with required shape and measure the inference time. Is there a similar function here? Thanks.

rickymedrano commented 3 years ago

+1, would also like to be able to test theoretical inference. Tried using the .rt file with trtexec but it doesn't work.

mive93 commented 3 years ago

Hi @ysyyork and @rickymedrano , you can use the test_rtinference, e.g., with yolov4 and 1 batch:

    ./test_rtinference yolo4_fp32.rt 1
ysyyork commented 3 years ago

Hi @ysyyork and @rickymedrano , you can use the test_rtinference, e.g., with yolov4 and 1 batch:

    ./test_rtinference yolo4_fp32.rt 1

Thanks @mive93 ! this is very helpful!