Faster_rcnn_inception_resnet_v2_atrous_coco Tests Really Slow

[etatbak]

Hello I trained my data with faster_rcnn_inception_resnet_v2_atrous_coco but when I try to test, it predicts really slow. I can test one image more than 25-30 seconds. I have 128Gb Rams and Nvidia quadro p5000. Also while testing it doesn't use gpu, am i wrong or it is normal to use only cpu? ( I already have tensorflow-gpu and trained my model with gpu).

[etatbak]

I think the problem is, each time when I run "def run_inference_for_single_image(image, graph):" it takes a lot time. I wrote a code for multiple images and I only call once inference grapy then I added all images to a list then I can return the list of "output_dicts". It only takes long when I run the codes once then it tests the images quickly.

[amussell]

I have had this issue as well, and I found it was due to instantiating a tf.Session every time run_inference_for_single_image was called. I augmented my code by wrapping the run_inference_for_single_image function into a class. When the class is instantiated I instantiate a session and reuse it every time run_inference_for_single_image is called. Processing the first image is always slow, but subsequent calls are fast.