tflite_runtime on Windows 10 raises a ValueError: Failed to load delegate from edgetpu.dll

[cgalleguillosm]

Hello everyone, I'm trying to run the following example using a Coral m2 module:

python examples/classify_image.py --model test_data/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite --labels test_data/inat_bird_labels.txt --input test_data/parrot.jpg

but the following error is raised:

Traceback (most recent call last):
  File "C:\Program Files\Python38\lib\site-packages\tflite_runtime\interpreter.py", line 152, in load_delegate
    delegate = Delegate(library, options)
  File "C:\Program Files\Python38\lib\site-packages\tflite_runtime\interpreter.py", line 111, in __init__
    raise ValueError(capture.message)
ValueError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "examples/classify_image.py", line 84, in <module>
    main()
  File "examples/classify_image.py", line 61, in main
    interpreter = make_interpreter(*args.model.split('@'))
  File "C:\Program Files\Python38\lib\site-packages\pycoral\utils\edgetpu.py", line 66, in make_interpreter
    delegates = [load_edgetpu_delegate({'device': device} if device else {})]
  File "C:\Program Files\Python38\lib\site-packages\pycoral\utils\edgetpu.py", line 42, in load_edgetpu_delegate
    return tflite.load_delegate(_EDGETPU_SHARED_LIB, options or {})
  File "C:\Program Files\Python38\lib\site-packages\tflite_runtime\interpreter.py", line 154, in load_delegate
    raise ValueError('Failed to load delegate from {}\n{}'.format(
ValueError: Failed to load delegate from edgetpu.dll

python -c "from ctypes import *; cdll.LoadLibrary('edgetpu.dll'); print('success')"
success

print(tflite_runtime.__git_version__)
48c3bae94a8b324525b45f157d638dfd4e8c3be1

Please help me.

[AkkiSony]

Unfortunately, my PC is also WINDOWS 10! I am connecting it directly to the PC, no USB hub.

[hjonnala]

Then it's perfect, I would suggest to you try it on own PC with edgetpu_runtime_20201204

[AkkiSony]

@hemanthreddyjonnala with edgetpu_runtime_20201204, it worked fine. The demo functioned today in the morning. But now for my custom object detection model, I got an error saying that I require runtime version(14). Hence, I upgraded to 20210726 version which is runtime version 14.

The below output is with runtime version(13):- edgetpu_runtime_20201204

(pycoral_venv) C:\pycoral_venv\Scripts\coral\pycoral>python examples\detect_image.py --model 3-classes-training\model_quantized_edgetpu.tflite --labels 3-classes-training\labels.txt.txt --input 3-classes-training\holes-test-input.jpg --output holes-output.jpg Traceback (most recent call last): File "examples\detect_image.py", line 108, in main() File "examples\detect_image.py", line 74, in main interpreter.allocate_tensors() File "C:\pycoral_venv\lib\site-packages\tflite_runtime\interpreter.py", line 408, in allocate_tensors return self._interpreter.AllocateTensors() RuntimeError: Failed to prepare for TPU. Failed precondition: Package requires runtime version (14), which is newer than this runtime version (13).Node number 2 (EdgeTpuDelegateForCustomOp) failed to prepare.

Please find the above error that I encountered. Lookling at this error, I chaned it to the newer version 20210726

[hjonnala]

The possible solutions are:

Compile the model with runtime version 13 by specifying -m flag (https://coral.ai/docs/edgetpu/compiler/#usage) example: ! edgetpu_compiler -s -m 13 ssdlite_mobiledet_dog_vs_cat.tflite

or

If the model(any specific operations) requires runtime 14 try it on coral dev board

or try it on different operating system either linux or mac OS as the model requires edgetpu runtime 14

[AkkiSony]

@hjonnala Thanks for always giving me some suggestion. Really happy about it! :) I will try specifying the model with edgetpu run time version 13.

If the model(any specific operations) requires runtime 14

I used https://colab.research.google.com/github/google-coral/tutorials/blob/master/retrain_detection_qat_tf1.ipynb to train my object detection model. So i dont know if this has any operations required for runtime version 14.

If this doesnt work, I wil start to work with dev board! :)

[AkkiSony]

@hjonnala I just wanted to update you that, even the demo is not working with this edge tpu runtime - 20210726. Demo worked only with 20201204 So, I think my model should not be the problem!

[hjonnala]

edge tpu runtime - 20210726 is same as edgetpu runtime 14. so, it does not work on windows as of now.

[johncblacker]

I am connecting to pc direct via USB 3 port.

Aug 10, 2021 8:30:29 AM Hemanth-Jonnala @.***>:

Then it's perfect, I would suggest to you try it on own PC with edgetpu_runtime_20201204

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub[https://github.com/google-coral/tflite/issues/46#issuecomment-895987903], or unsubscribe[https://github.com/notifications/unsubscribe-auth/AB6WAASJE2HEDPJPS26HQGTT4ELWJANCNFSM43NLV3WQ]. Triage notifications on the go with GitHub Mobile for iOS[https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675] or Android[https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email]. [data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAYAAABzenr0AAAAAXNSR0IArs4c6QAAAARzQklUCAgICHwIZIgAAAAaSURBVFiF7cEBAQAAAIIg/69uSEABAAAA7wYQIAABRwGgiAAAAABJRU5ErkJggg==###24x24:true###][Tracking image][https://github.com/notifications/beacon/AB6WAAUPYT3KYCIQJQPVWWLT4ELWJA5CNFSM43NLV3W2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOGVT3BPY.gif]

[AkkiSony]

! edgetpu_compiler -s -m 13 ssdlite_mobiledet_dog_vs_cat.tflite

@hjonnala A brilliant idea! :) Thank a lot. It worked for me! :) But, the accuracy is very bad. Is there a way where I can measure the accuracy of the model now? While trainng the model, hthe tensorboard did not work for me. Hence, I do not know how to measure the accuracy of the model now. However, I used Average Precision matrix and when loss was around 0.7, I stopped trainig. The object detection on the PC using google colab was better than now. Is there a reason?

BUT THANK YOU SO MUCH FOR THE HACK! :)

[hjonnala]

you can refer to these tutorials: https://github.com/google-coral/tutorials

This section might help: https://colab.research.google.com/github/google-coral/tutorials/blob/master/retrain_classification_ptq_tf2.ipynb#scrollTo=RMLYBDe_e849

Please open new issues as these questions are not relevant to this thread.

[puresilk]

I've stumbled into this erorr, and have found out an additional piece of information which might be valuable to the Coral dev team.

Naive execution of the bat file (by double clicking it) will NOT install UsbDk.

I identified the problem as being an asterisk reference to UsbDk, and patched it to the correct path (as in edgetpu_runtime_20210119, the last version which works for me)

echo Installing UsbDk
:: start /wait msiexec /i "%ROOTDIR%\third_party\usbdk\UsbDk_*.msi" /quiet /qb! /norestart
start /wait msiexec /i "%ROOTDIR%\third_party\usbdk\UsbDk_1.0.22_x64.msi" /quiet /qb! /norestart

This WILL transform the ValueError: Failed to load delegate from edgetpu.dll to

$ pyenv exec python examples/classify_image.py --model test_data/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite --labels test_data/inat_bird_labels.txt --input test_data/parrot.jpg
Segmentation fault

I have documented this more thoroughly here in a blogpost:

[hjonnala]

@puresilk please try these steps to install runtime 14 on windows: https://github.com/google-coral/pycoral/issues/91#issuecomment-1264524697