Open alexw9988 opened 2 years ago
Also, here are the versions for gasket
and apex
on the Dev Board in case that's interesting:
Hi @alexw9988 can you please share model compilation logs.edgetpu_compiler -s model.tflite
@hjonnala Here you go:
Edge TPU Compiler version 16.0.384591198
Started a compilation timeout timer of 180 seconds.
Model compiled successfully in 91877 ms.
Input model: /content/gdrive/MyDrive/models/new/protbuf_white_onelabel_v2_full_size/model_uint8.tflite
Input size: 3.57MiB
Output model: /content/gdrive/MyDrive/models/new/protbuf_white_onelabel_v2_full_size/edgetpuTEST/model_uint8_edgetpu.tflite
Output size: 26.09MiB
On-chip memory used for caching model parameters: 3.99MiB
On-chip memory remaining for caching model parameters: 2.33MiB
Off-chip memory used for streaming uncached model parameters: 0.00B
Number of Edge TPU subgraphs: 1
Total number of operations: 90
Operation log: /content/gdrive/MyDrive/models/new/protbuf_white_onelabel_v2_full_size/edgetpuTEST/model_uint8_edgetpu.log
Operator Count Status
ADD 10 Mapped to Edge TPU
LOGISTIC 1 Mapped to Edge TPU
CONCATENATION 5 Mapped to Edge TPU
PAD 4 Mapped to Edge TPU
DEPTHWISE_CONV_2D 17 Mapped to Edge TPU
TRANSPOSE_CONV 5 Mapped to Edge TPU
CONV_2D 46 Mapped to Edge TPU
QUANTIZE 2 Mapped to Edge TPU
Compilation child process completed within timeout period.
Compilation succeeded!
@alexw9988 Most probably its due to model input size(256x3072x3). Can you try with reduced model input size. Please check this link for example input sizes: https://coral.ai/models/all/
If there are no concerns with the model please share the input CPU tflite model. Thanks.
@hjonnala It did in fact work with smaller input sizes (i.e. 256x256), however in theory there should not be any issues with larger models, right?
Here's the model: model_white_256x3072_v2.tflite.zip
Thanks a lot!
I am able to run benchmark tests on dev board. Can you please share the test scripts that you are working with.
Edit: Now i am able to see HIB errors occasionally.
https://github.com/hjonnala/snippets/blob/main/devboard/single_model_benchmark
@hjonnala Glad I'm not the only one ;) Do you have any idea what's going on?
Hi @alexw9988 according to the error status value 0x8, it's an internal chip AXI bus write response error. Which suggests that there's some physical HW problem. It might be that chip is defective or it's having clock and/or power issues.
I think, its due to clock issues as it is happening occasionally. Please make sure you are providing 5V/3A power supply to the dev board.
@hjonnala I'm using a proper 5V/3A power supply and the board is just a few weeks old - so I suppose there's not really much I can do ...
Would it help to try and reduce the CPU clock speed a bit?
Hi @alexw9988, @hjonnala I have the same issue, HIB error with status 0x8 Did you find any solution?
@liv-kuka can you please share the model here and let us know on which Coral Product are you running the model? Thanks!
I'm using Coral Dev Board, with yolov5s model.
The model was exported using export.py
from yolo project and then compiled with edgetpu_compliter
.
Trying to run inference with detect.py
from yolo project (also tried with detect.py from google-coral/examples-camera
repo but got error)
If I run model with input size 480x480 or 640x640 I get HIB error.
But with model 416x416 and lower inference running fine.
Note: I have been using 2 Dev Boards and on one of them HIB error always appears earlier.
640x640 model: yolov5s-int8_edgetpu.zip
HIB error with status 0x8
This error in Dev Board is mostly related to stress due to extensive i/o operations. Please reduce the model input size or change the model itself.
Description
Hi all!
I'm using the Google Coral Dev Board and am experiencing very inconsistent results when trying to run simple benchmarking tests on quantized and converted segmentation models.
I am using a custom retrained MobilenetV2 256x3072xRGB with segmentation head. Sometimes it runs successful, however some other times it throws this error on invocation:
The issue is, I have no way to consistently reproduce the error. I can run my testing script several times in a row and roughly one in 10 tries will be successful, the others will result in an error containing the above lines.
I attached 3 different verbose log outputs where it failed and one where it succeeded. Note how the 3 different fail logs have vastly different line counts. The program execution will just hang forever and I had to kill the terminal whenever it got to the errors.
success.log fail3.log fail2.log fail.log
Can anyone tell me what's going on? Also, I'm using the latest runtime version (14) and used the latest compiler version when creating the edgetpu model:
On Dev Board:
On Colab Notebook where compilation takes place:
Click to expand!
### Issue Type Bug, Support ### Operating System Mendel Linux ### Coral Device Dev Board ### Other Devices _No response_ ### Programming Language Python 3.7 ### Relevant Log Output _No response_