search

scheckmedia / keras-shufflenet

ShuffleNet Implementation using Keras Functional Framework 2.0
MIT License
77 stars 40 forks source link

Execution time for forward pass #8

Closed afzalxo closed 5 years ago

afzalxo commented 5 years ago

Hey! Can you do me a favor and check the execution time of a single forward pass for batch size = 1? I am not getting nearly what is stated in the paper of about 37.8ms for ShuffleNet 1x (g=3). Even though I an using ShuffleNet 1x (g=8), I am getting over 1.35s per inference iteration.

Edit x1: Thanks! Edit x2: I am using an Intel Xeon E5-2680 v3 @ 2.50GHz which is a fairly decent processor. The paper states the benchmarks were performed on a Qualcomm Snapdragon 820 (single thread) which we should easily be able to surpass given we are using desktop/server CPUs.

scheckmedia commented 5 years ago

please check the url of #3

afzalxo commented 5 years ago

Thanks! I was only measuring the execution time of the first session.run which is always much slower compared to subsequent runs according to this. Btw, can you please suggest a link on how you did the timing and profiling as in the URL you suggested?

scheckmedia commented 5 years ago

Yes the warm-up time is much slower because of the tensor initialization, I think.

Tensorflow comes with its own benchmark tool: https://github.com/tensorflow/tensorflow/tree/master/tensorflow/tools/benchmark There you will find all necessary steps to build and execute this tool.

Keep in mind that you have to freeze the graph for the evaluation.

afzalxo commented 5 years ago

Thanks! I am actually using my own tf based implementation which is only that of the forward pass, using pre-trained model from the tensorpack models. Resolved!