Example parrot classification inference takes 1800ms on Windows

[talcs]

Description

PS C:\programs\coral\pycoral> python .\examples\classify_image.py --model .\test_data\mobilenet_v2_1.0_224_inat_bird_quant.tflite --labels .\test_data\inat_bird_labels.txt --input .\test_data\parrot.jpg
----INFERENCE TIME----
Note: The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory.
1822.0ms
1819.0ms
1804.6ms
1810.8ms
1791.1ms
-------RESULTS--------
Ara macao (Scarlet Macaw): 0.79297

1. I followed the installation instructions from the docs , which currently include the edgetpu_runtime_20210726 package.
2. As I suffered from the infamous ValueError: Failed to load delegate from edgetpu.dll error, I downgraded the edgetpu_runtime package: I uninstalled the runtime package I had and instead, installed one-before-the-latest version, which is edgetpu_runtime_20210119.
3. Then it finally worked, yet the runtimes appear to be dramatically slower than expected, and not just the first inference.
4. My USB port appears to be USB3, as it has 9 pins.

Python packages versions:

>>> import numpy as np
>>> import tflite_runtime
>>> import pycoral
>>> np.__version__
'1.21.2'
>>> tflite_runtime.__version__
'2.5.0.post1'
>>> pycoral.__version__
'2.0.0'

The Python version is Python 3.7.4, 64bit.

I will appreciate any advice, Thanks!

Click to expand!

### Issue Type Performance ### Operating System Windows 10 ### Coral Device USB Accelerator ### Other Devices _No response_ ### Programming Language Python 3.7 ### Relevant Log Output _No response_

[hjonnala]

Here are the results on my windows machine. The accuracy results are seem to be different. Can you please share the model you are using?

C:\Users\heman\coral\pycoral>py examples/classify_image.py --model test_data/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite --labels test_data/inat_bird_labels.txt --input test_data/parrot.jpg
----INFERENCE TIME----
Note: The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory.
102.6ms
14.8ms
14.8ms
12.9ms
15.8ms
-------RESULTS--------
Ara macao (Scarlet Macaw): 0.75781

[talcs]

It appears that we run exactly the same command - I shared my run command in the first comment. Should I also upload the model file?

Note that the model was downloaded automatically, by running bash examples/install_requirements.sh classify_image.py, as guided here. Since it is a Windows machine, I used Cygwin to run the bash script, but first had to get rid of all \r characters in the examples/install_requirements.sh file, to avoid syntax errors. Don't know if that matters.

[hjonnala]

Hi, Inference time might varies from system to system but accuracy should be same. The model accuracy has to be 0.75781. Can you please upload your model since you are getting 0.79297 accuracy.

Thanks

[hjonnala]

I haven't seen this properly. You are trying CPU tflite model mobilenet_v2_1.0_224_inat_bird_quant.tflite. Can you please try with mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite

[talcs]

Thanks, this solved my problem! Didn't know that there were two files in that directory and that only one of them was intended for the TPU. 😃

[google-coral-bot[bot]]

Are you satisfied with the resolution of your issue? Yes No