search

NVIDIA-AI-IOT / tf_trt_models

TensorFlow models accelerated with NVIDIA TensorRT
BSD 3-Clause "New" or "Revised" License
682 stars 245 forks source link

Slower performance when writing to file? #36

Open atyshka opened 5 years ago

atyshka commented 5 years ago

I'm using the jupyter example to create these trt-optimized graphs for use in my projects. I'm taking the TensorRT converted graph, writing it to a file, and then loading that pb file in and performing inference. However, I've noticed the runtimes I get when doing this are about 3 times greater than the runtimes reported by the notebook. Either the notebook is reporting incorrect times or somehow reconstructing the graph from the file creates a different graph than the original that is somehow slower. Has anyone been able to reproduce this issue?

atyshka commented 5 years ago

For example, for resnet_50_fpn_coco, sess.run(...) takes about 0.1 s, but when using the graph written to the .pb file the same code takes 0.3s.