search

Robert-JunWang / Pelee

Pelee: A Real-Time Object Detection System on Mobile Devices
Apache License 2.0
885 stars 255 forks source link

SSD+MobileNet： FPS #2

Open MrLinNing opened 6 years ago

MrLinNing commented 6 years ago

How do you measure FPS？

Robert-JunWang commented 6 years ago

The FPS of intel is got by Caffe time. The FPS of iPhone is calculated by the average time of 100 images processed by the CoreML model. This time includes the image pre-processing time, but it does not include the time of the post-processing part (decoding the bounding-boxes and performing non-maximum suppression). Usually, post-processing is done on the CPU, which can be executed asynchronously with the other parts that are executed on mobile GPU. Hence, the actual speed should be very close to our test result.

foralliance commented 6 years ago

@Robert-JunWang

There seem to be two ways of calculating FPS.

1.via the 'time' model of caffe with batch_size = 1：

build/tools/caffe time -model=models/VGGNet/VOC0712/refinedet_vgg16_320x320/deploy.prototxt -gpu=0

2.get the time interval and estimate the FPS. for example, got the following lines after I run examples/ssd/score_ssd_pascal.py:

I0713 20:50:25.756376 117599 net.cpp:684] Ignoring source layer mbox_loss
I0713 20:51:32.472930 117599 solver.cpp:531] Test net output #0: detection_eval = 0.723217
I0713 20:51:32.473057 117599 solver.cpp:325] Optimization Done.

The total time spent on evaluating 4952 test images is about 67 seconds, and thus FPS = 4952 / 67 = 74.

Which method should be used to calculate FPS? Is there any other way?

Robert-JunWang commented 6 years ago

I think both methods are fine, as long as you evaluate all the models in the same way.

foralliance commented 6 years ago

@Robert-JunWang thanks your reply The difference between the two approaches is a bit big.

ujsyehao commented 6 years ago

@foralliance what is batch_size in your method2? I think batch size affect speed much.