search

google-coral / pycoral

Python API for ML inferencing and transfer-learning on Coral devices
https://coral.ai
Apache License 2.0
347 stars 144 forks source link

poor classification results with edgetpu model #50

Closed staebchen0 closed 2 years ago

staebchen0 commented 2 years ago

Description

I create my models with Teachable Machine. For this test, I downloaded all three model variants.

So far I have been using the unquant.tflite model for classification, which achieves excellent results. Test with the script: images_predict_unquant_Teach_Lite.py - All images in a directory are checked Model: model_unquant.tflite

To speed things up, I got the coral usb accelerator bought to classify with an edge tpu model

Unfortunately I get bad results with the edgetpu model and only two classes are returned as a result (none and cat), regardless of which class I test Test with script: classify_image_edgetpu.py - only individual images are checked Model: example_edgetpu_v13.tflite

Reference issues: https://github.com/google-coral/libedgetpu/issues/29#issuecomment-909207700 https://github.com/google-coral/pycoral/issues/49 https://github.com/google-coral/pycoral/issues/48

The zipfile contains the two scripts, all models and a test dataset per class of 292 edgetpu_issue_Windows10 (3).zip images

I have not yet used the model_quantized.tflite model for the tests. However, it is included as a zip file

Install edgetpu env

$ pip freeze
edgetpu @ https://dl.google.com/coral/edgetpu_api/edgetpu-2.14.0-cp37-cp37m-win_amd64.whl
install==1.3.4
numpy==1.21.2
opencv-contrib-python==4.5.3.56
opencv-python==4.5.3.56
Pillow==8.3.2
pycoral @ https://github.com/google-coral/pycoral/releases/download/v2.0.0/pycoral-2.0.0-cp37-cp37m-win_amd64.whl
tflite-runtime @ https://github.com/google-coral/pycoral/releases/download/v2.0.0/tflite_runtime-2.5.0.post1-cp37-cp37m-win_amd64.whl

Install tflite env 

$ pip freeze
absl-py==0.11.0
astunparse==1.6.3
cachetools==4.2.0
certifi==2020.12.5
chardet==4.0.0
flatbuffers==1.12
gast==0.3.3
google-auth==1.24.0
google-auth-oauthlib==0.4.2
google-pasta==0.2.0
grpcio==1.32.0
h5py==2.10.0
idna==2.10
importlib-metadata==3.3.0
imutils==0.5.3
Keras-Preprocessing==1.1.2
Markdown==3.3.3
numpy==1.19.5
oauthlib==3.1.0
opencv-python==4.5.1.48
opt-einsum==3.3.0
Pillow==8.1.0
Pillow-PIL==0.1.dev0
protobuf==3.14.0
pyaes==1.6.1
pyasn1==0.4.8
pyasn1-modules==0.2.8
requests==2.25.1
requests-oauthlib==1.3.0
rsa==4.6
scipy==1.6.0
six==1.15.0
Telethon==1.19.0
tensorboard==2.4.0
tensorboard-plugin-wit==1.7.0
tensorflow==2.4.0
tensorflow-estimator==2.4.0
termcolor==1.1.0
typing-extensions==3.7.4.3
urllib3==1.26.2
Werkzeug==1.0.1
wrapt==1.12.1
zipp==3.4.0
Click to expand! ### Issue Type Performance ### Operating System Windows 10 ### Coral Device USB Accelerator ### Other Devices _No response_ ### Programming Language Python 3.7 ### Relevant Log Output _No response_
staebchen0 commented 2 years ago

one more hint I think I forgot in the script images_predict_unquant_Teach_Lite.py the line of code 363

remove_Folder(imgPathOut)

comment out. This will always delete the result. And you can't see the result.

staebchen0 commented 2 years ago

attached is an example of the edgetpu model

All images in the specified directory are checked. images_predict_edgetup_all.zip Here you can see very clearly that the classification does not work

staebchen0 commented 2 years ago

I now also have a test with the model: model_quantized.tflite made.

I previously complied the model with the edgetpu complier in version 13.

The result is definitely better, all with the edgetup model

how can that be?

The hit rate is best with model_unquant.tflite

result in the attached file Result classification.ods

hjonnala commented 2 years ago

can you please replace model_quantized_edgetpu.tflite with version 13 edgetpu file and run classification_cpu_vs_edgetpu.py.

After running please check comaparision_results.csv and share the results.

Thanks

pycoral_50.tar.gz

staebchen0 commented 2 years ago

i don't have pandas installed. Which version should I install in the env?

hjonnala commented 2 years ago

please install pandas==1.1.5

staebchen0 commented 2 years ago

Since I use Windows 10, I unfortunately cannot use the current runtime, but have to use the version edgetpu_runtime_20210119.zip to use.

That's why I get the error when loading the model_quantized_edgetpu.tflite model Error: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

here the result classification_cpu_vs_edgetpu.py comaparision_results.csv

hjonnala commented 2 years ago

If we look at the results column both quantized tflite output and edgetpu tflite outputs are similar.

can you please tell what is the problem here?

staebchen0 commented 2 years ago

When creating the result file, with classification_cpu_vs_edgetpu.py the paths were wrong. I have to adjust them again from: models = [model for model in os.listdir(base_dir) if model.endswith('.tflite')] in models = [model for model in os.listdir(model_dir) if model.endswith('.tflite')]

and from model_path = os.path.join(base_dir, model) in model_path = os.path.join(model_dir, model

now the models that are to be checked are actually loaded in the path:

model As you can see in the screenshot, the other Tflite models should have appeared in the csv file. Unfortunately, I now get the error: Python was not found; run without arguments to install from the Microsoft Store, or disable this shortcut from Settings > Manage App Execution Aliases.

As described above, I had tested the three models and the edetpu model did not give the correct results https://github.com/google-coral/pycoral/issues/50#issuecomment-927155975

staebchen0 commented 2 years ago

I found the mistake

cmd = f'''python3 classify_image.py \ in cmd = f'''python classify_image.py \

i used a virtual environment. That would have to be added to your script. How can I specify the correct virtual environment here?

                cmd = f'''python classify_image.py \
                  --model {model_path}\
                  --labels {os.path.join(labels_dir, 'labels.txt')}\
                  --input {os.path.join(images_dir, dir, image)}\
                  --count 1 --top_k 4'''
hjonnala commented 2 years ago

for example if your env is pycoral_venv try to replace python with <path to pycoral_venv/Scripts/python.exe

staebchen0 commented 2 years ago

The script is now running. with the supplied model, the error occurs - model_quantized_edgetpu.tflite this is created with version 14.

this is how it looks now:

import os
import subprocess
import pandas as pd

def classification_tests(base_dir, images_dir):
    rows = []
    models = [model for model in os.listdir(model_dir) if model.endswith('.tflite')]
    images_dirs = os.listdir(images_dir)
    for dir in images_dirs:
        images = os.listdir(os.path.join(images_dir, dir))
        for image in images[:10]:
            print(dir, image)
            for model in models:

                model_path = os.path.join(model_dir, model)
                cmd = f'''envCoralPy8\Scripts\python.exe classify_image.py \
                  --model {model_path}\
                  --labels {os.path.join(labels_dir, 'labels.txt')}\
                  --input {os.path.join(images_dir, dir, image)}\
                  --count 1 --top_k 4'''
                proc = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, )
                output = proc.communicate()[0].decode('utf-8')
                lines = output.split('\n')
                for line in lines[3:]:
                    if line != '':
                        rows.append([model, line.split(':')[0], line.split(':')[1]])

    df = pd.DataFrame(rows, columns=['model', 'inferecne', 'results'])
    df.to_csv(os.path.join(base_dir, f'comaparision_results.csv'), index=False)
    print(df)

    return df

base_dir = os.path.dirname(os.path.abspath(__file__))
model_dir = os.path.join(base_dir, 'model')
labels_dir = os.path.join(base_dir, 'model')
images_dir = os.path.join(base_dir, 'dataset')
classification_df = classification_tests(base_dir, images_dir)

Error:

cat 0.jpg
"__main__" geladen
"runpy" geladen
Frame skipped from debugging during step-in.
Note: may have been skipped because of "justMyCode" option (default == true). Try setting "justMyCode": false in the debug configuration (e.g., launch.json).
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

cat 1.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

cat 10.jpg
Das Programm "python.exe" wurde mit Code 0 (0x0) beendet.

I throw the model out of the directory and test it again
hjonnala commented 2 years ago

can you please replace model_quantized_edgetpu.tflite with version 13 edgetpu file and run classification_cpu_vs_edgetpu.py.

After running please check comaparision_results.csv and share the results.

Thanks

pycoral_50.tar.gz

staebchen0 commented 2 years ago

here first the results with my models.

Unfortunately, the values ​​are no longer correctly transferred to the columns.

I'll try the model now: model_quantized_edgetpu.tflit [comaparision_results.csv](https://github.com/google-coral/pycoral/files/7246035/comaparision_results.csv) e

staebchen0 commented 2 years ago

result with your model: model_quantized_edgetpu.tflite

that can't work because I can't use the new version 14 under Win10. As already described above. csv file is empty

Messages

cat 0.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

cat 1.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

cat 10.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

cat 100.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

cat 101.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

cat 102.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

cat 103.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

cat 104.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

cat 105.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

cat 106.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

mouse 0.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

mouse 1.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

mouse 10.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

mouse 100.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

mouse 102.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

mouse 103.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

mouse 104.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

mouse 105.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

mouse 106.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

mouse 107.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

none 1500.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

none 1501.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

none 1502.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

none 1503.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

none 1504.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

none 1505.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

none 1506.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

none 1507.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

none 1508.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

none 1509.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

other 3815.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

other 3816.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

other 3817.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

other 3818.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

other 3819.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

other 3820.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

other 3824.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

other 3825.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

other 3826.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

other 3827.jpg
Traceback (most recent call last):
  File "classify_image.py", line 88, in <module>
    main()
  File "classify_image.py", line 62, in main
    interpreter.allocate_tensors()
  File "C:\Users\anja-\Anja_Programme\AnjaCoral\envCoralPy8\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

"__main__" geladen
"runpy" geladen
Empty DataFrame
Columns: [model, inferecne, results]
Index: []
Der Thread 'MainThread' (0x1) hat mit Code 0 (0x0) geendet.
Das Programm "python.exe" wurde mit Code 0 (0x0) beendet.
hjonnala commented 2 years ago

I know, v14 does not work on your machine. we are only interested in comparison between quantized cpu tflite model and corresponding v13 edgetpu model. Please let us know, if there are issues with results comparison for these two models.

staebchen0 commented 2 years ago

so here the results of the models:

example_edgetpu_v13 quantized.tflite example_edgetpu_v13.tflite

In the results file you can see that with the example_edgetpu_v13 only 2 classes are returned and that the classification is wrong file comaparision_results.csv

hjonnala commented 2 years ago

if you look at example_edgetpu_v13.tflite using https://netron.app/ there are only 2 output tensors [1 2]. That's why you are getting only 2 results.

image

staebchen0 commented 2 years ago

that's strange. As described, I created all three models with a teachable machine in one run. And then exported in the various variants. Then I compiled it into version 13.

two models work only the edgetpu model shows differences.

Here you can see that there were 4 classes in the original model edgetpu. Then this must have happened when compiling to version 13 orgEdgetpu

I have all 3 models with help https://colab.research.google.com/github/google-coral/tutorials/blob/master/compile_for_edgetpu.ipynb#scrollTo=joxrIB0I3cdi compiled ! dgetpu_compiler -s -m 13 $ TFLITE_FILE

staebchen0 commented 2 years ago

here the original edgetpu file converted_edgetpu.zip

hjonnala commented 2 years ago

I am not sure how you get the edgetpu file with two output tensors.. It should not happen if input TFLITE_FILE has 4 output tensors.

staebchen0 commented 2 years ago

could you complieren the model in version 13?

i then test how the classification works

hjonnala commented 2 years ago

Please check attachment and just keepmodel_quantized_v13.tflite(quatized cpu tflite model) and model_quantized_v13_edgetpu.tflite(v13 edgetpu model) and test it.. pycoral_50.tar.gz

staebchen0 commented 2 years ago

the result of your model: model_quantized_v13_edgetpu.tflite is very good.

See file. comaparision_results.csv

how did you compile the model into version 13?

hjonnala commented 2 years ago

! edgetpu_compiler -s -m 13 model_quantized_v13.tflite

hjonnala commented 2 years ago

Hi, do you have questions here, if not please close this ticket.

staebchen0 commented 2 years ago

somehow the worm is in there. I wanted to compile the original Edge tpu file to version 13, as I did before. The compiler message: Edge TPU compiler

my guess is now as follows. The compiler compiled the file on the last attempt and two classes were removed from the edgetpu model. There is no other explanation.

And the file from you model_quantized_v13_edgetpu.tflite didn't come from the edgetpu.tflite model, right?

and you have not compiled the model_edgetpu.tflite in version 13

hjonnala commented 2 years ago

I wanted to compile the original Edge tpu file to version 13, as I did before. The compiler message: Edge TPU compiler

you can't compile edgeptu tflite file using compiler to version 13. The input for compiler is always uncompiled tflite file..

my guess is now as follows. The compiler compiled the file on the last attempt and two classes were removed from the edgetpu model. There is no other explanation.

Compiler does not change the model architecture(It won't remove any classes). If the input tflite file has 2 output tensors then output would also 2 output tensors..

And the file from you model_quantized_v13_edgetpu.tflite didn't come from the edgetpu.tflite model, right?

The input for model_quantized_v13_edgetpu.tflite is model_quantized_v13.tflite. model_quantized_v13.tflite is just a copy of model_quantized.tflite

and you have not compiled the model_edgetpu.tflite in version 13

I don't know about model_edgetpu.tflite. Th only input I have used from your zip folder is model_quantized.tflite

google-coral-bot[bot] commented 2 years ago

Are you satisfied with the resolution of your issue? Yes No