search

google-coral / pycoral

Python API for ML inferencing and transfer-learning on Coral devices
https://coral.ai
Apache License 2.0
351 stars 145 forks source link

Unsupported data type in custom op handler #1

Closed alex-moffitt closed 3 years ago

alex-moffitt commented 3 years ago

I'm running Python3.7 docker image with libedgetpu1-max (tried std as well) and as of yesterday I am no longer able to find the python3-pycoral package (which seemed to work while it was there).

I have since installed pycoral-1.0.0-cp37-cp37m-linux_aarch64.whl and tflite_runtime-2.5.0-cp37-cp37m-linux_aarch64.whl and am now getting the following error:

Unsupported data type in custom op handler: -1579713216Node number 2 (EdgeTpuDelegateForCustomOp) failed to prepare.

teraSurfer commented 3 years ago

I'm also facing the same issue with raspberry pi 4, I've tried it with the new tflite_runtime-2.5.0-cp37-cp37m-linux_armv7l.whl and I'm using libedgetpu1-std version 14.1. Any help on this will be great.

Namburger commented 3 years ago

Hi guys, We recently have a release of libedgetpu which may have caused some mismatches. Could you share the outputs of this command:

$ dpkg -l | grep libedgetpu
teraSurfer commented 3 years ago 
ii  libedgetpu1-std:armhf                 14.1                                  armhf        Support library for Edge TPU
Namburger commented 3 years ago

@teraSurfer Can you try a an apt update and reinstall libedgetpu? Basically, your version should be 15.0 for it to be compatible with tflite_runtime 2.5.

teraSurfer commented 3 years ago

@Namburger I tried it but I'm not seeing version 15.0 available anywhere. https://packages.cloud.google.com/apt/dists/coral-edgetpu-stable/main/binary-armhf/Packages only shows version 14.1

15.0 is in unstable. Should I try it anyway?

a-rich commented 3 years ago

Same issue here -- also not seeing where version 15.0 is:

$ sudo dpkg -l | grep edge
ii  libedgetpu1-std:armhf                14.1                                         armhf        Support library for Edge TPU

$ pip freeze | grep tflite
tflite-runtime @ https://github.com/google-coral/pycoral/releases/download/release-frogfish/tflite_runtime-2.5.0-cp37-cp37m-linux_armv7l.whl

$ python3
Python 3.7.3 (default, Jul 25 2020, 13:03:44)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import dill
>>>dill.load_session('~/coral_inference_test_session_linux.pkl')
>>> test()
2020.11.04.tflite
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<ipython-input-4-a0f03a08574a>", line 4, in test
  File "<ipython-input-2-334f4462c6f6>", line 3, in init
  File "/home/iap/.local/lib/python3.7/site-packages/tflite_runtime/interpreter.py", line 259, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Internal: Unsupported data type in custom op handler: 34030992Node number 3 (EdgeTpuDelegateForCustomOp) failed to prepare.
sacefe commented 3 years ago

same issue , and not seeing version 15

$ dpkg -l | grep libedgetpu
ii  libedgetpu1-std:amd64                      14.1                                                                        amd64        Support library for Edge TPU

$ python3 classify_image.py --model models/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite --labels models/inat_bird_labels.txt --input images/parrot.jpg
Traceback (most recent call last):
  File "classify_image.py", line 122, in <module>
    main()
  File "classify_image.py", line 100, in main
    interpreter.allocate_tensors()
  File "/home/saguilar/.local/lib/python3.6/site-packages/tflite_runtime/interpreter.py", line 259, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Internal: Unsupported data type in custom op handler: 46686256Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.
Namburger commented 3 years ago

Hi all, sorry about that, it seems like new package isn't uploaded to our apt repo for armv7l and arm64, but it is for x86_64.

For now, please download the new libedgetpu from this link and unzip it: 

$ https://dl.google.com/coral/edgetpu_api/edgetpu_runtime_20201105.zip
$ unzip edgetpu_runtime_20201105.zip

and then points LD_LIBRARY_PATH to the path of the new share library when you run the example (for instance, this is how I ran it on the Dev Board):

$ LD_LIBRARY_PATH=/home/mendel/edgetpu_runtime/libedgetpu/direct/aarch64 python3 classify_image.py   --model models/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite   --labels models/inat_bird_labels.txt   --input images/parrot.jpg
----INFERENCE TIME----
Note: The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory.
16.6ms
3.8ms
3.0ms
3.3ms
4.1ms

Will update you guys as soon as we get the new package uploaded on the apt repo.

teraSurfer commented 3 years ago

I downloaded (15.0) from the unstable link and it is working fine. Thank you!

Namburger commented 3 years ago

@teraSurfer thanks for confirming and apologies, we'll have the new version reup soon

sacefe commented 3 years ago

@Namburger in windows 10, I installed the zip file and created the environment variable for LD_LIBRARY_PATH However, when I run the command, it hangs forever. Any help on this will be great. Note: I tested in Linux, and it worked ok, but I need it in Windows 10

AustinDeric commented 3 years ago

I downloaded the link provided here for the libedgetpu: https://github.com/google-coral/pycoral/issues/1#issuecomment-724312390

I also used the test_data submodule in this repo: https://github.com/google-coral/test_data/tree/c21de4450f88a20ac5968628d375787745932a5a

$ LD_LIBRARY_PATH=/home/my_username/libs/coral/edgetpu_runtime/libedgetpu/direct/aarch64 python3.8 examples/classify_image.py --model test_data/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite --labels test_data/inat_bird_labels.txt --input test_data/parrot.jpg
Traceback (most recent call last):
  File "examples/classify_image.py", line 82, in <module>
    main()
  File "examples/classify_image.py", line 60, in main
    interpreter.allocate_tensors()
  File "/home/my_username/.local/lib/python3.8/site-packages/tflite_runtime/interpreter.py", line 242, in allocate_tensors
    return self._interpreter.AllocateTensors()
  File "/home/my_username/.local/lib/python3.8/site-packages/tflite_runtime/interpreter_wrapper.py", line 76, in AllocateTensors
    return _interpreter_wrapper.InterpreterWrapper_AllocateTensors(self)
RuntimeError: Internal: Unsupported data type in custom op handler: 0Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.
$ dpkg -l | grep libedgetpu
ii  libedgetpu1-max:amd64                           15.0                                 amd64        Support library for Edge TPU

UPDATE: It works, i had a different version of tflite-runtime installed in my user space.

$ pip3 freeze | grep tflitetflite-runtime==2.5.0
tflite-runtime==2.5.0
$ LD_LIBRARY_PATH=/home/my_username/libs/coral/edgetpu_runtime/libedgetpu/direct/aarch64 python3.8 examples/classify_image.py --model test_data/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite --labels test_data/inat_bird_labels.txt --input test_data/parrot.jpg
----INFERENCE TIME----
Note: The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory.
11.1ms
2.7ms
2.6ms
2.6ms
2.6ms
-------RESULTS--------
Ara macao (Scarlet Macaw): 0.75781
jingw222 commented 3 years ago

Maybe a bit off-topic, but how can I rollback to libedgetpu1-std=14.1 now that it seems v15.0 is the only available deb package from the official repository? Are there archived versions hosted somewhere?

Namburger commented 3 years ago

@jingw222 yes, you can visit coral.ai/software, there'll be a section with the edgetpu tarballs including release date and installation script.

Another way is to install python3-edgetpu which will install a "legacy" libedgetpu package from our apt repo

jingw222 commented 3 years ago

@jingw222 yes, you can visit coral.ai/software, there'll be a section with the edgetpu tarballs including release date and installation script.

Another way is to install python3-edgetpu which will install a "legacy" libedgetpu package from our apt repo

Thank you, Nam! Just managed to install a legacy libedgetpu-std package via the second option, which was quite interestingly also v15.0. In fact, I could have just installed the package directly by a different name as libedgetpu1-legacy-std. The two packages live side by side.

Namburger commented 3 years ago

@jingw222 yes, that's correct, it's an attempt for s to make sure that users doesn't get the wrong configurations and run into issues

openedhardware commented 3 years ago

Hi all, sorry about that, it seems like new package isn't uploaded to our apt repo for armv7l and arm64, but it is for x86_64.

For now, please download the new libedgetpu from this link and unzip it:

$ https://dl.google.com/coral/edgetpu_api/edgetpu_runtime_20201105.zip
$ unzip edgetpu_runtime_20201105.zip

and then points LD_LIBRARY_PATH to the path of the new share library when you run the example (for instance, this is how I ran it on the Dev Board):

$ LD_LIBRARY_PATH=/home/mendel/edgetpu_runtime/libedgetpu/direct/aarch64 python3 classify_image.py   --model models/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite   --labels models/inat_bird_labels.txt   --input images/parrot.jpg
----INFERENCE TIME----
Note: The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory.
16.6ms
3.8ms
3.0ms
3.3ms
4.1ms

Will update you guys as soon as we get the new package uploaded on the apt repo.

Hi, @Namburger

When do you expect the armv7l and arm64 uploaded to the apt repo?

Namburger commented 3 years ago

Hello @openedhardware its all uploaded by now, if you install python3-pycoral package, everything should be updated!

seanreynoldscs commented 3 years ago

Hey, I'm also seeing this error. File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 154, in load_delegate raise ValueError('Failed to load delegate from {}\n{}'.format( ValueError: Failed to load delegate from libedgetpu.so.1

I see: $ dpkg -l | grep libedgetpu rc libedgetpu1-max:arm64 15.0 arm64 Support library for Edge TPU ii libedgetpu1-std:arm64 15.0 arm64 Support library for Edge TPU

and: $ pip3 freeze | grep tflite-runtime tflite-runtime==2.5.0

So I'm not sure what is wrong. These seem to be the versions that people were expecting.

Thoughts?

Namburger commented 3 years ago

@seanreynoldscs try this:

sudo usermod -aG plugdev $USER
sudo reboot now

if it still doesn't work, I suggest sending an strace:

strace python3 your_script.py > strace.log
seanreynoldscs commented 3 years ago

Umm... Wow... okay so can you explain that ? :) it worked. +1 for super quick feedback too!

Namburger commented 3 years ago

Nice, that means you were blocked by the linux user permission, normally this is already allowed by default so some users doesn't have this problems!

Luke1962 commented 3 years ago

Hi, me too with this error, after migrating yesterday from python3-edgetpu to python3-coral libraries (see https://coral.ai/software/) My platform : Arm + Ubuntu 18.04 + Python 3.6.9 ( + ROS Melodic) I was using this model : /mobilenet_ssd_v2_face_quant_postprocess_edgetpu.tflite, which was perfectly working with previous python3-edgetpu lib. Steps to reproduce:

pi@rockpi:~$ python3
Python 3.6.9 (default, Oct  8 2020, 12:12:24) 
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pycoral
>>> from pycoral.utils import edgetpu
>>> DET_MODEL_PATH = '/home/pi/tflite.models/mobilenet_ssd_v2_face_quant_postprocess_edgetpu.tflite'
>>> interpreter = edgetpu.make_interpreter(DET_MODEL_PATH)
>>> interpreter.allocate_tensors()

_Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/dist-packages/tflite_runtime/interpreter.py", line 242, in allocate_tensors
    return self._interpreter.AllocateTensors()
  File "/usr/local/lib/python3.6/dist-packages/tflite_runtime/interpreter_wrapper.py", line 115, in AllocateTensors
    return _interpreter_wrapper.InterpreterWrapper_AllocateTensors(self)
RuntimeError: Internal: Unsupported data type in custom op handler: 0Node number 2 (EdgeTpuDelegateForCustomOp) failed to prepare._

BUT, if I change the model to ssd_mobilenet_v1_1_metadata_1. ( from example found ), no more error occurs:

>>> DET_MODEL_PATH = '/home/pi/tflite.models/ssd_mobilenet_v1_1_metadata_1.tflite'
>>> interpreter = edgetpu.make_interpreter(DET_MODEL_PATH) 
>>> interpreter.allocate_tensors()
>>>

So are the new libraries no more compatible with previous models (*_edgetpu.tflite)? If yes is this behavior by design or backward compatibility could be recovered ?

By the way it would be very helpfully to have a "Rosetta" style example showing how to migrate from python3-edgetpu to python3-coral libraries

Namburger commented 3 years ago

@Luke1962 model should still works. Besides from the pythonpackage, you'll also need to install the new libdgetpu.so: https://coral.ai/software#debian-packages

seanreynoldscs commented 3 years ago

Okay new issue. I swapped the USBC for the M.2 B+M key. What is weird is I'm having the same issue as before, but now the command you gave me doesnt work.

$ python3 examples/classify_image.py --model test_data/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite --labels test_data/inat_bird_labels.txt --input test_data/parrot.jpg Traceback (most recent call last): File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 152, in load_delegate delegate = Delegate(library, options) File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 111, in init raise ValueError(capture.message) ValueError

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "examples/classify_image.py", line 84, in main() File "examples/classify_image.py", line 61, in main interpreter = make_interpreter(*args.model.split('@')) File "/usr/lib/python3/dist-packages/pycoral/utils/edgetpu.py", line 66, in make_interpreter delegates = [load_edgetpu_delegate({'device': device} if device else {})] File "/usr/lib/python3/dist-packages/pycoral/utils/edgetpu.py", line 42, in load_edgetpu_delegate return tflite.load_delegate(_EDGETPU_SHARED_LIB, options or {}) File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 154, in load_delegate raise ValueError('Failed to load delegate from {}\n{}'.format( ValueError: Failed to load delegate from libedgetpu.so.1

When following instructions from: https://coral.ai/docs/dev-board-mini/get-started/#7-run-a-model-using-the-pycoral-api

Thoughts?