Why does quantified model run slower?

guichristmann / edge-tpu-tiny-yolo

Run Tiny YOLO-v3 on Google's Edge TPU USB Accelerator.

MIT License

102 stars 31 forks source link

Why does quantified model run slower? #2

Open oroelipas opened 4 years ago

oroelipas commented 4 years ago

I know these models are designed to run in Coral USB but should't also it run faster in a PC?

It takes about 1.15 seconds to run a tiny-yolov3.tflite in my coumputer but arround 15 seconds to run the quant_coco-tiny-v3-relu.tflite

Is this normal behaviour? will it run faster than 1.15s in Coral USB TPU when I buy it?

Thanks for answering!

guichristmann commented 4 years ago

I don't know the reason, but I also got a bad performance when running the quantized model on the desktop CPU. However, it runs fine/as expected with the Edge TPU.

parthjdoshi commented 3 years ago

@guichristmann What kind of inference speeds did you get with YOLOv3-tiny? In terms of FPS?

AkkiSony commented 3 years ago

@oroelipas @parthjdoshi @guichristmann Hi. I could convert my yolov3 model to quantized tflite model. But when I try to run the inference.py script. I do not see any detection happening on the input image. Were you able to get it doneusing this platform? Did you make any changes in the script? Please do help me get this run. Thank you in advance guys.

lukqw commented 3 years ago

@parthjdoshi I get around 16-17 fps with a yolov3-tiny model with relu as the activation function on device with edge tpu