search

ardianumam / Tensorflow-TensorRT

This repository is for my YT video series about optimizing a Tensorflow deep learning model using TensorRT. We demonstrate optimizing LeNet-like model and YOLOv3 model, and get 3.7x and 1.5x faster for the former and the latter, respectively, compared to the original models.
303 stars 110 forks source link

Performance about TensorRT #13

Open zhangchbin opened 5 years ago

zhangchbin commented 5 years ago

I tried to accelerate my TensorFlow code by using TensorRT, but it didn't get any improvement? in frozen_model, num of all_nodes = 893 in TensorRT_model, num of trt_engine_nodes = 0 in TensorRT_model, num of all_nodes = 831 Is there anything wrong? Thanks!

ardianumam commented 5 years ago

I ever encountered such problem in Jetson TX2. My solution was updating Tensorflow version to 1.11.

zhangchbin commented 5 years ago

I ever encountered such problem in Jetson TX2. My solution was updating Tensorflow version to 1.11.

Actually, I used cuda10.0, cudnn7.4.3, Tensorflow 1.13. Is there anything wrong?

ardianumam commented 5 years ago

I'm not sure. Maybe you can try to optimize another model first, e.g., in code 1 in this repo, to make sure all settings work.

PythonImageDeveloper commented 5 years ago

Hi @ardianumam @godman2016 Can you solved this problem? I optimized my trained model (1 class), the ssdlite_mobilenetv2 and ssd_resnet50, with TensorRT, but the performance did't improve significantly, I reach from 0.12 sec to 0.11 sec on GTX 1080 ti, why? I installed Tensorflow 1.12.0 , cuda 9 , TensorRT 4.0.1.6 packages on Ubuntu 16.04.

zhangchbin commented 5 years ago

Hi @ardianumam @godman2016 Can you solved this problem? I optimized my trained model (1 class), the ssdlite_mobilenetv2 and ssd_resnet50, with TensorRT, but the performance did't improve significantly, I reach from 0.12 sec to 0.11 sec on GTX 1080 ti, why? I installed Tensorflow 1.12.0 , cuda 9 , TensorRT 4.0.1.6 packages on Ubuntu 16.04.

I haven't solved this problem. Maybe it can accelerate some models, not all.

ardianumam commented 5 years ago

Hi @ardianumam @godman2016 Can you solved this problem? I optimized my trained model (1 class), the ssdlite_mobilenetv2 and ssd_resnet50, with TensorRT, but the performance did't improve significantly, I reach from 0.12 sec to 0.11 sec on GTX 1080 ti, why? I installed Tensorflow 1.12.0 , cuda 9 , TensorRT 4.0.1.6 packages on Ubuntu 16.04.

I haven't solved this problem. Maybe it can accelerate some models, not all.

I also tried to perform TRT optimization several days ago to SSD MobileNet1 with 1 class. I got 45 FPS in Jetson TX2 for both before & after TRT optimization. My temporary conclusion is: (i) TRT is less optimized for network like MobileNet, maybe due its separable convolution that already performs very small computation so that there is less space for optimization. (ii) When I use more classes (e.g., 80 classes in COCO), there is more difference after TRT optimization (TRT seems optimizing conv. operation for the output prediction which is proportional to the number of classes).