Coral USB failing to accelerate custom model

[puelon]

I have a Transfer Learning model from a VGG16 that I trained on my thermal images and after doing PQAT to try and compress model size, I notice that it still has a large inference time of around 5 seconds per image. The same model being compressed to an intel neural stick 2 has around 450ms. I suspect that my model may not be compatible layer-wise with the coral usb device. This is the link to my tflite converted model: https://www.mediafire.com/file/e3z930wlh4vkwt8/model_91_test_acc_PQAT.tflite/file It has two outputs and this is the layer summary:

block1_conv1 (Conv2D)       (None, 128, 160, 64)      1792      

 block1_conv2 (Conv2D)       (None, 128, 160, 64)      36928     

 block1_pool (MaxPooling2D)  (None, 64, 80, 64)        0         

 block2_conv1 (Conv2D)       (None, 64, 80, 128)       73856     

 block2_conv2 (Conv2D)       (None, 64, 80, 128)       147584    

 block2_pool (MaxPooling2D)  (None, 32, 40, 128)       0         

 block3_conv1 (Conv2D)       (None, 32, 40, 256)       295168    

 block3_conv2 (Conv2D)       (None, 32, 40, 256)       590080    

 block3_conv3 (Conv2D)       (None, 32, 40, 256)       590080    

 block3_pool (MaxPooling2D)  (None, 16, 20, 256)       0         

 block4_conv1 (Conv2D)       (None, 16, 20, 512)       1180160   

 block4_conv2 (Conv2D)       (None, 16, 20, 512)       2359808   

 block4_conv3 (Conv2D)       (None, 16, 20, 512)       2359808   

 block4_pool (MaxPooling2D)  (None, 8, 10, 512)        0         

 block5_conv1 (Conv2D)       (None, 8, 10, 512)        2359808   

 block5_conv2 (Conv2D)       (None, 8, 10, 512)        2359808   

 block5_conv3 (Conv2D)       (None, 8, 10, 512)        2359808   

 block5_pool (MaxPooling2D)  (None, 4, 5, 512)         0         

 flatten (Flatten)           (None, 10240)             0         

 dense (Dense)               (None, 16)                163856    

 output (Dense)              (None, 1)                 17     

I wonder if this is something with the model or is it a hardware issue?

[hjonnala]

Hi @puelon I am getting an inference about 53ms on my Machine. Pleases try the below steps:

wget https://raw.githubusercontent.com/hjonnala/snippets/main/coral_inference.py
python3 coral_inference.py -m /home/Downloads/model_91_test_acc_PQAT_edgetpu.tflite

[puelon]

Please forgive me, due to the amount of optimization on my code I actually forgot to run the command to compile the model for Edge TPU, it is now running around 80ms on my Rpi

[google-coral-bot[bot]]

Are you satisfied with the resolution of your issue? Yes No