Update pycoral for Python 3.11 and modern versions of Tensorflow

feranick commented 7 months ago

Description

Pycoral is available only with support for Python 3.9. Most distribution now no longer ship with that version, rather either 3.10 or 3.11. So there is factually no version of pycoral available to download. Furthermore, building pycoral from source is not compatible with Python 3.10 or 3.11 (again, not supported) and relies on an ancient version of tensorFlow (2.5.0) to work. While this al points to a lack of maintenance, it would be great to at least update so it can be used with, well, pretty much any modern version of OS.

Click to expand!

### Issue Type Build/Install ### Operating System Linux, Ubuntu, Mac OS ### Coral Device USB Accelerator, M.2 Accelerator A+E, M.2 Accelerator B+M, M.2 Accelerator with dual Edge TPU ### Other Devices Raspberry Pi 3, Rapsberry Pi 4 ### Programming Language Other ### Relevant Log Output _No response_

tranzmatt commented 5 months ago

Has this project been officially abandoned?

feranick commented 5 months ago

There hasn't been any comment, action, or even statement in a very long time. The repos are frozen in time. Such a pity when every chip manufacturer is presenting NPUs as the next big thing, and Google (who pioneered it with the edgeTPU so long ago) decided to drop the ball. To make it worse, no official statement either, and a community that relies/invested on it is left it to its own devices (pun very much intended). And to make it worse, coral.ai still sells these products as fully supported when they are nothing but.

Sad, and infuriating.

Emojibotbot commented 5 months ago

I'm experiencing the same issue on 64 bit Raspberry Pi OS 12.4 running on a Pi 4 which has python 3.11.2. I tried to build pycoral myself and have not managed to make things work yet.

I guess all the layoffs have hit this team pretty hard.

feranick commented 5 months ago

I ended up forking the libedgetpu and bringing it up to date with compatibility to the latest version of TF. This is all I really needed (as I don't use pycoral, but directly access the edgeTPU withtflite_runtime) so I can fully use the edgeTPU. I took a look at pycoral but it seems that a whole lot more work is needed to bringing it up to date.

https://github.com/feranick/libedgetpu

Emojibotbot commented 5 months ago

I ended up forking the libedgetpu and bringing it up to date with compatibility to the latest version of TF. This is all I really needed (as I don't use pycoral, but directly access the edgeTPU withtflite_runtime) so I can fully use the edgeTPU. I took a look at pycoral but it seems that a whole lot more work is needed to bringing it up to date.

https://github.com/feranick/libedgetpu

I was able to install the edgetpu runtime without issues on my rig using instructions here (but not have tried to run it in anyway yet.) What made you need to update?

feranick commented 5 months ago

I was able to install the edgetpu runtime without issues on my rig (but not have tried to run it in anyway yet.) What made you need to update?

That is the point. The provided whl install fine if you have the correct Python environment (see below), but they segfault if you are running tlite_runtime > 2.11.1.

The wheels provided are only available for up to Python 3.9, su pretty useless in any modern Linux release (Raspbian/Debian-bookworm current uses 3.11, ubuntu 22.04 uses 3.10). It may work with "legacy" debian-bullseye which uses Python 3.9.

feranick commented 5 months ago

Plenty of discussion here:

https://github.com/google-coral/edgetpu/issues/812

feranick commented 5 months ago

There is also another issue: the google-dkms gasket is incompatible with linux kernel >= 6.5 and therefore the edgeTPU won't be recognized. The solution is a recompile of the gasket itself, following instructions here:

https://github.com/google-coral/edgetpu/issues/820

Emojibotbot commented 5 months ago

I see you have done a lot of work to update things already & nice job! I thought the issue was just that pycoral was not up to date, but the lack of updates much more systemic across all of google-coral.

Emojibotbot commented 5 months ago

Is it safe to assume your fork would not work? https://github.com/feranick/pycoral

feranick commented 5 months ago

Correct. If it will ever work, I will post binaries there, but I have limited time to spend on it at the moment.

Thanks

On 2/16/24 12:21 PM, Emojibotbot wrote:

Is it safe to assume your fork would not work? https://github.com/feranick/pycoral

— Reply to this email directly, view it on GitHub https://github.com/google-coral/pycoral/issues/137#issuecomment-1948939758, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAIY77VBR5V37GYBMFAVQDYT6IQLAVCNFSM6AAAAABAT3RXKCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNBYHEZTSNZVHA. You are receiving this because you authored the thread.Message ID: @.***>

--------------uAcACjLxWMBrvKW4lt9XPhZx Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit

<!DOCTYPE html>

Correct. If it will ever work, I will post binaries there, but I have limited time to spend on it at the moment.

Thanks

On 2/16/24 12:21 PM, Emojibotbot wrote:

Is it safe to assume your fork would not work? https://github.com/feranick/pycoral

—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.

--------------uAcACjLxWMBrvKW4lt9XPhZx--

feranick commented 4 months ago

Thanks to @pkgoogle and @namburger, the libdgetpu repo/library has been updated, a big step forward. Next that should be updated is libcoral/pycoral, the frontend libraries that are (supposedly) the standard way to interact with the edgeTPU. My fork still does not compile, and several issues are at play. There is an open issue/PR for TF that is supposed to fix one of the issues: Change visibility status for lite/schema/schema_conversion_utils to allow compilation of external libraries. If there is a way for this PR to be merged, that'd be great, since in the current state, such visibility issue in TF (which is downloaded by Bazel in tmp_folder during the compilation of libcoral) hangs compilation of pycoral.

feranick commented 4 months ago

I finally managed to successfully compile libcoral through this new fork of mine.

I submitted a PR request: https://github.com/google-coral/libcoral/pull/36

Of course, this needs the visibility issue to be fixed first. Yet, compilation can be successfully done manually, until the issue above is fixed. Two steps:

Change last line in WORKSPACE from:
```
path = "/usr/system_libs",
```
to
```
path = "/",
```
as the former is designed when operating with Docker.
Edit: nano ~/.cache/bazel/_bazel_XXXX/XXXXXXX/external/org_tensorflow/tensorflow/lite/schema/BUILD and remove visibility = [":utils_friends"],

from both:

cc_library(
    name = "schema_utils",
    srcs = ["schema_utils.cc"],
    hdrs = ["schema_utils.h"],
    compatible_with = get_compatible_with_portable(),
    visibility = [":utils_friends"],
    deps = [
        ":schema_fbs",
        "//tensorflow/lite/kernels/internal:compatibility",
        "@flatbuffers//:runtime_cc",
    ],
)

cc_library(
    name = "schema_conversion_utils",
    srcs = ["schema_conversion_utils.cc"],
    hdrs = ["schema_conversion_utils.h"],
    compatible_with = get_compatible_with_portable(),
    visibility = [":utils_friends"],
    deps = [
        ":schema_fbs",
        "//tensorflow/lite/kernels/internal:compatibility",
        "@flatbuffers",
    ],
)

feranick commented 4 months ago

A new fork for pycoral is now available, which relies on the libedgetpu and libcoral forks, and it has been tested to work. All three libraries are currently targeting the upcoming TensorFlow version 2.16.0 (currently in rc0) status, with the hope that the visibility issue is fixed in that release. Once TF 2.16.0 is officially released as stable, I will create a PR for the updated libedgetpu (which now targets TF2.15.0 in the official repo). Upon merging libedgetpu into the official repo (which currently targets TF 2.15.0), another PR is ready for the updated libcoral. Once that is merged, you guessed it, I will push a PR for pycoral based on this fork.

In the meantime, one can already build libedgetpu from the official repo (against TF 2.15.0) or the fork needed for this upcoming version of pycoral. With that, pycoral can be built as instructed in the README file in the forked repo, with the only caveat that the visibility issue needs to be manually addressed, as detailed in the comment above..

Thanks again to @namburger and @pkgoogle for pushing forward the effort to modernize the EdgeTPU libraries.

feranick commented 4 months ago

A new fork for pycoral is now available, which relies on the libedgetpu and libcoral forks, and it has been tested to work. All three libraries are currently targeting the upcoming TensorFlow version 2.16.0 (currently in rc0) status, with the hope that the visibility issue is fixed in that release. Once TF 2.16.0 is officially released as stable, I will create a PR for the updated libedgetpu (which now targets TF2.15.0 in the official repo). Upon merging libedgetpu into the official repo (which currently targets TF 2.15.0), another PR is ready for the updated libcoral. Once that is merged, you guessed it, I will push a PR for pycoral based on this fork.

In the meantime, one can already build libedgetpu from the official repo (against TF 2.15.0) or the fork needed for this upcoming version of pycoral. With that, pycoral can be built as instructed in the README file in the forked repo, with the only caveat that the visibility issue needs to be manually addressed, as detailed in the comment above..

Thanks again to @Namburger and @pkgoogle for pushing forward the effort to modernize the EdgeTPU libraries.

adelmiljkovic commented 3 months ago

[A new fork for pycoral is now available]

I tested it and seems like something is wrong. When I run one of the examples I get INFO: Created TensorFlow Lite XNNPACK delegate for CPU. and I did some test and the inference times are the same for running the edgetpu models compared to the regular ones (e.g. movenet_single_pose_lightning_ptq_edgetpu.tflite and movenet_single_pose_lightning_ptq.tflite)

Anyone else experiencing this?

feranick commented 3 months ago

Which version of libedgetpu are you using? What platform?

adelmiljkovic commented 3 months ago

Which version of libedgetpu are you using? What platform?

I'm running a RaspberryPi 4 (Bookworm) and libedgetpu1-max_16.0tf2.16.1-1.bookworm_arm64

feranick commented 3 months ago

Can you provide a usable script to try this out? Better yet, can you run one of the official examples available here? https://coral.ai/docs/edgetpu/tflite-python/#inferencing-example

feranick commented 3 months ago

Note that with pycoral, the tflite model is used directly (i.e. there is no need to convert it into _edgetpu.tflite), so it is possible that the runtime is pretty much the same. If you can, you should try to run the tflite and _egetpu.tflite independently without using pycoral, but directly from tensorflow.

feranick commented 3 months ago

Also keep in mind that the fork is still experimental due to compatibility issues in TF 2.16.1.

adelmiljkovic commented 3 months ago

I haven't had the chance to delve into using TFLite directly without PyCoral yet, but I'm genuinely intrigued. I'll be giving it a try tomorrow and see how it pans out. All the work you've put in is greatly appreciated. Thank you for your great contributions!

feranick commented 3 months ago

Thank you for trying. Using libedgetpu directly works smoothly and with no problems. As pycoral goes, a major issue will be resolved when TF 2.17.0 will be released, that is preventing compilation of pycoral. There are also other issues at play, especially w.r.t. armhf. In all honesty, at the moment, the pycoral is a hackjob because of this inconsistencies of so many different elements in needs to play along (TF, libedgetpu, libcoral, pycoral)... But we'll get there.

adelmiljkovic commented 3 months ago

Running TFLite directly works perfectly on RPi 4 with Bookworm and Python 3.11. Great job!👏👏

feranick commented 3 months ago

Great. Again, I'll keep working on pycoral, but that will take time. libedgetpu seems to be remaining well aligned with TF, so direct access to edgetpu through it should be the stable choice, for now.

julianmata commented 2 months ago

Hey @feranick, I am trying to install your pycoral build, I followed the guide here: https://github.com/feranick/pycoral/releases When I try with python version of 3.11 it works okay but when I try 3.10 builds I am getting this error: `Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] on linux Type "help", "copyright", "credits" or "license" for more information.

from pycoral.utils import edgetpu Traceback (most recent call last): File "", line 1, in File "/home/julian/.local/lib/python3.10/site-packages/pycoral/utils/edgetpu.py", line 24, in from pycoral.pybind._pywrap_coral import GetRuntimeVersion as get_runtime_version ImportError: Python version mismatch: module was compiled for Python 3.11, but the interpreter version is incompatible: 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0].`

Just to be correct I installed the driver like this: wget https://github.com/feranick/libedgetpu/releases/download/16.0TF2.16.1-1/libedgetpu1-std_16.0tf2.16.1-1.ubuntu22.04_amd64.deb sudo dpkg -i libedgetpu1-std_16.0tf2.16.1-1.ubuntu22.04_amd64.deb then installed runtime wheel like this: pip3 install https://github.com/feranick/TFlite-builds/releases/download/v.2.16.1/tflite_runtime-2.16.1-cp310-cp310-linux_x86_64.whl then installed pycoral wheel like this: pip3 install https://github.com/feranick/pycoral/releases/download/v2.0.1TF2.16.1/pycoral-2.0.1-cp310-cp310-linux_x86_64.whl after that I tried to use it as I have shown above, where I try to import edgetpu and I am getting that error. Am I missing something, and as a bonus can you provide your env setup for building the wheel?

Thank you, for any answer I would appreciate it!

feranick commented 2 months ago

PyCoral is currently not stable, until tflite_runtime reaches v 2.17.0 (with the next release of Tensorflow). You should wait until then.

EnziinSystem commented 2 months ago

@feranick

You are the hope for the Edge TPU community after Google showed irresponsibility by abandoning the project.

We have the right to ask because we bought the hardware from Google.

Currently, I have the following:

Raspberry Pi-5 with OS 64-bit, Kernel 6.6.30-v8+
AI Coral Edge TPU module from Pinboards
PiCamera module V3
Python 3.11

What steps do I need to take to be able to use TPU with Python 3.11 and avoid the maze of version conflicts?

Thank you.

feranick commented 2 months ago

I don't provide support, but will provide a list of commands to get it going as it worked for me for the USB version and the M2 version of the TPU on a Desktop linux.. It might not work for you, given it's a third-party module. So if it works, great, if not, you will have to figure out why.

mkdir ~/edgetpu-inst
cd edgetpu-inst
wget https://github.com/feranick/gasket-driver/releases/download/v1.0-18.1/gasket-dkms_1.0-18_all.deb
sudo apt install linux-headers-generic dkms linux-headers-6.1.0-21-arm64 linux-headers-6.1.0-21-common linux-headers-arm64 linux-kbuild-6.1
sudo dpkg -i gasket-dkms_1.0-18_all.deb
wget https://github.com/feranick/libedgetpu/releases/download/16.0TF2.16.1-1/libedgetpu1-std_16.0tf2.16.1-1.bookworm_arm64.deb
sudo dpkg -i libedgetpu1-std_16.0tf2.16.1-1.bookworm_arm64.deb
wget https://github.com/feranick/TFlite-builds/releases/download/v.2.16.1/tflite_runtime-2.16.1-cp311-cp311-linux_aarch64.whl
sudo mv /usr/lib/python3.11/EXTERNALLY-MANAGED /usr/lib/python3.11/EXTERNALLY-MANAGED.bkp
sudo pip3 install --upgrade tflite_runtime-2.16.1-cp311-cp311-linux_aarch64.whl

Next, I prepared a test script to see whether the setup is properly installed. You can download it as follow:

cd ~/edgetpu-inst/
wget https://github.com/feranick/TFlite-builds/releases/download/v.2.16.1/test_edgetpu.tar.xz
tar xvaf test_edgetpu.tar.xz
cd test_edgetpu

Now you can test if it works, by running: python3 test-edgetpu.py

if you get OK you are good to go. if not and you get an error message, well, there is an issue, of course, but I may not be able to assist as it is directly related to your setup.

A few notes:

If you are planning on using pycoral, that is currently broken, and will be fixed once TF is released in version 2.17.0.
This means you can use the TPU through tflite_runtime now on quantized tflite models (you can refer to the google documentation for details), and works great for me.
This assumes you have all your trained and quantized models. One is provided in the test-edgetpu.py for testing purposes only.

han-so1omon commented 2 months ago

@EnziinSystem Try the steps here if any issues arise. I had some issues with the Hat AI from Pineboards

EnziinSystem commented 2 months ago

@feranick

I followed your instructions and the results were as follows (with libedgetpu and tflite_runtime from your repos)

$ uname -a
Linux smarthome 6.6.28+rpt-rpi-v8 #1 SMP PREEMPT Debian 1:6.6.28-1+rpt1 (2024-04-22) aarch64 GNU/Linux

$ lspci -nn | grep 089a
0000:03:00.0 System peripheral [0880]: Global Unichip Corp. Coral Edge TPU [1ac1:089a]

$ ls -la /dev/apex_0
crw-rw---- 1 root apex 120, 0 May  9 15:03 /dev/apex_0

$ sudo lspci -vvv|grep -i MSI-X
        Capabilities: [d0] MSI-X: Enable- Count=128 Masked-
        Capabilities: [d0] MSI-X: Enable+ Count=9 Masked-
        Capabilities: [b0] MSI-X: Enable+ Count=61 Masked-

Because I cannot use kernel 6.6.30 , I recompile DTB with msi-parent = <0x6d>

Run demo

$ python3 test_libedgetpu.py
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/tflite_runtime/interpreter.py", line 166, in load_delegate
    delegate = Delegate(library, options)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/tflite_runtime/interpreter.py", line 104, in __init__
    raise ValueError(capture.message)
ValueError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/developer/coral/test_edgetpu/test_libedgetpu.py", line 12, in <module>
    model = tflite.Interpreter(model_path='model_classifier_CNN_edgetpu.tflite', experimental_delegates=[tflite.load_delegate(edgeTPUSharedLib,{})])
                                                                                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/tflite_runtime/interpreter.py", line 168, in load_delegate
    raise ValueError('Failed to load delegate from {}\n{}'.format(
ValueError: Failed to load delegate from libedgetpu.so.1

Check libedgetpu

 $ ls /usr/lib/aarch64-linux-gnu/ | grep libedgetpu
libedgetpu.so.1
libedgetpu.so.1.0

Although there are still errors, I still thank you for your help.

adelmiljkovic commented 2 months ago

@EnziinSystem I think that might have to do something with your specific setup. I run a Pi4 (Bookworm) and Coral over USB and using libedgetpu and the tflite runtime (without pycoral) worked literally on the first try.

EnziinSystem commented 2 months ago

@EnziinSystem I think that might have to do something with your specific setup. I run a Pi4 (Bookworm) and Coral over USB and using libedgetpu and the tflite runtime (without pycoral) worked literally on the first try.

If I run via Docker and Python 3.9, I also run it from the first time. The problem here is that Python 3.11 and the latest Kernels 6.6.30 and TPU cards from PCI have compatibility issues.

feranick commented 2 months ago

The issue is with the PCIe HAT. gaster-dkms has support for kernels >6.8 and Python 3.11 is proven to work fine with the binaries provided. I'd recommend reaching out for the maker of the HAT as it is clearly an issue they should be provide support for. Again, I used the USB edgeTPU with Python 3.11 and kernel 6.6.28 (the officially supported one, any other from rpi-update is not recommended for stability and daily usage as recommended by rpi-update itself) and the M.2 version of the edgeTPU on Dektop Debian running a similar setup.

EnziinSystem commented 2 months ago

After following the instructions at Pinboards

I can use TPU M2 card on Kernel 6.6.30 with dtoverlay=pineboards-hat-ai instead of recompiling DTB and change ms-parent

$ uname -a
Linux local 6.6.30-v8+ #1762 SMP PREEMPT Thu May  9 15:17:51 BST 2024 aarch64 GNU/Linux

Install TFlite - Update libedgetpu1 from @feranick repos

wget https://github.com/feranick/libedgetpu/releases/download/16.0TF2.16.1-1/libedgetpu1-std_16.0tf2.16.1-1.bookworm_arm64.deb

sudo dpkg -i libedgetpu1-std_16.0tf2.16.1-1.bookworm_arm64.deb

wget https://github.com/feranick/TFlite-builds/releases/download/v.2.16.1/tflite_runtime-2.16.1-cp311-cp311-linux_aarch64.whl

sudo mv /usr/lib/python3.11/EXTERNALLY-MANAGED /usr/lib/python3.11/EXTERNALLY-MANAGED.bkp

sudo pip3 install --upgrade tflite_runtime-2.16.1-cp311-cp311-linux_aarch64.whl

Run a demo

https://github.com/google-coral/tflite/tree/master/python/examples/classification

$ python3 classify_image.py \
  --model models/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite \
  --labels models/inat_bird_labels.txt \
  --input images/parrot.jpg
/home/developer/coral/google-coral/tflite/python/examples/classification/classify_image.py:103: DeprecationWarning: ANTIALIAS is deprecated and will be removed in Pillow 10 (2023-07-01). Use LANCZOS or Resampling.LANCZOS instead.
  image = Image.open(args.input).convert('RGB').resize(size, Image.ANTIALIAS)
----INFERENCE TIME----
Note: The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory.
12.5ms
2.3ms
2.3ms
2.3ms
2.3ms
-------RESULTS--------
Ara macao (Scarlet Macaw): 0.77734

Note: We still can't use Pycoral like @feranick said.

UPDATE:

I installed pycoral from @feranick repos

wget https://github.com/feranick/pycoral/releases/download/v2.0.1TF2.16.1/pycoral-2.0.1-cp311-cp311-linux_aarch64.whl

pip3 install pycoral-2.0.1-cp311-cp311-linux_aarch64.whl

And run example Pycoral

https://github.com/google-coral/pycoral/tree/master/examples

$ python3 examples/classify_image.py \
  --model test_data/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite  \
  --labels test_data/inat_bird_labels.txt \
  --input test_data/parrot.jpg
----INFERENCE TIME----
Note: The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory.
12.9ms
2.7ms
2.7ms
2.7ms
2.7ms
-------RESULTS--------
Ara macao (Scarlet Macaw): 0.75781

It seems that Pycoral also worked in the latest system versions or I haven't encountered a situation where Pycoral failed.

EnziinSystem commented 2 months ago

@feranick

Today 05/13/2024, TensorFlow Lite released 2.17 and I can build it successfully on Pi-5.

pip3 install pybind11

git clone https://github.com/tensorflow/tensorflow.git

PYTHON=python3 tensorflow/tensorflow/lite/tools/pip_package/build_pip_package_with_cmake.sh native

Since you mentioned this before, what improvements or problems does tflite-2.17 have for pycoral?

feranick commented 2 months ago

Tensorflow (including lite) v2.17 has not been released, as it is not showing in the official releases:

https://github.com/tensorflow/tensorflow/releases

Usually Google first does 1-3 releases candidates, which are also not showing. What you have is a dev tree that may lead at some point to 2.17. But is NOT 2.17 stable.

Once 2.17 is out, pycoral can finally be compiled for it in a way that it is compatible with tensorflow. It's minor detail due to an internal library. There are of course other issues, but until 2.17 is actually out nothing happens.

feranick commented 2 months ago

If you are interested in the details follow here:

https://github.com/google-coral/libcoral/pull/36

and here:

https://github.com/google-coral/libedgetpu/pull/60

feranick commented 2 months ago

And of course, if you want to use the 2.17 dev branch, you can compile libedgetpu yourself against it, and possibly attempt pycoral as well as indicated in my fork:

https://github.com/feranick/pycoral

But I don't provide any support for it (as it might crash and burn). Once 2.17 I will start again trying to merge pycoral into upstream with all changes that are needed.

withanhdammit commented 1 month ago

I'm running a desktop version of Debian 12 using the 6.6.13 kernel from bookworm-backports

uname -a
Linux frigate 6.6.13+bpo-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.6.13-1~bpo12+1 (2024-02-15) x86_64 GNU/Linux

My Coral TPU is this one (PCIe Dual Edge TPU) , and my computer does recognize it's installed (sadly only one of the cores, but that's OK)

lspci | grep Coral
06:00.0 System peripheral: Global Unichip Corp. Coral Edge TPU

I modified your steps slightly to use the amd64/x86_64 versions in your repo

mkdir ~/edgetpu-inst
cd edgetpu-inst
wget https://github.com/feranick/gasket-driver/releases/download/v1.0-18.1/gasket-dkms_1.0-18_all.deb
sudo apt install linux-headers-generic dkms linux-headers-6.1.0-21-amd64 linux-headers-6.1.0-21-common linux-headers-amd64 linux-kbuild-6.1
sudo dpkg -i gasket-dkms_1.0-18_all.deb
wget https://github.com/feranick/libedgetpu/releases/download/16.0TF2.16.1-1/libedgetpu1-std_16.0tf2.16.1-1.bookworm_amd64.deb
sudo dpkg -i libedgetpu1-std_16.0tf2.16.1-1.bookworm_amd64.deb
wget https://github.com/feranick/TFlite-builds/releases/download/v.2.16.1/tflite_runtime-2.16.1-cp311-cp311-linux_x86_64.whl
sudo mv /usr/lib/python3.11/EXTERNALLY-MANAGED /usr/lib/python3.11/EXTERNALLY-MANAGED.bkp
sudo pip3 install --upgrade tflite_runtime-2.16.1-cp311-cp311-linux_x86_64.whl

when I run the 'test_libedgetpu.py' I get:

~/edgetpu-inst/test_edgetpu$ python3 test_libedgetpu.py 
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/tflite_runtime/interpreter.py", line 166, in load_delegate
    delegate = Delegate(library, options)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/tflite_runtime/interpreter.py", line 104, in __init__
    raise ValueError(capture.message)
ValueError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/darhl/edgetpu-inst/test_edgetpu/test_libedgetpu.py", line 12, in <module>
    model = tflite.Interpreter(model_path='model_classifier_CNN_edgetpu.tflite', experimental_delegates=[tflite.load_delegate(edgeTPUSharedLib,{})])
                                                                                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/tflite_runtime/interpreter.py", line 168, in load_delegate
    raise ValueError('Failed to load delegate from {}\n{}'.format(
ValueError: Failed to load delegate from libedgetpu.so.1

At this point, I'm a little over my head, any suggestions for where to look to resolve this?

Edit: Looks like it was a permissions issue. When I run the test with sudo, it works perfectly.

~/edgetpu-inst/test_edgetpu$ sudo python3 test_libedgetpu.py 
OK

EnziinSystem commented 1 month ago

@withanhdammit

You should compile the driver from source instead of installing from the deb file.

git clone https://github.com/google/gasket-driver.git

cd gasket-driver

sudo debuild -us -uc -tc -b

Go back to the parent directory and install the built package:

cd ..

sudo dpkg -i gasket-dkms_1.0-18_all.deb

The installation process also requires linux-header files.

Camilochiang commented 1 month ago

Hei @EnziinSystem . I have a situation with a pineberry hat (the old version, from january). I can run my models in models but randomly the TPU hang and freeze after reading it. Have you experience something similar? Thanks!

EnziinSystem commented 1 month ago

Hei @EnziinSystem . I have a situation with a pineberry hat (the old version, from january). I can run my models in models but randomly the TPU hang and freeze after reading it. Have you experience something similar? Thanks!

After using HAT AI Coral and installing the operating system on the SSD, I ran a simple example of the Picamera2 library, I also experienced that sometimes the system paused for a while and then continued running.

https://github.com/raspberrypi/picamera2/blob/main/examples/tensorflow/real_time_with_labels.py

This example does not use TPU but only inferences based on Tensoflow Lite.

I don't know the reason why.

withanhdammit commented 1 month ago

@withanhdammit

You should compile the driver from source instead of installing from the deb file.
git clone https://github.com/google/gasket-driver.git

cd gasket-driver

sudo debuild -us -uc -tc -b

Go back to the parent directory and install the built package:

cd ..

sudo dpkg -i gasket-dkms_1.0-18_all.deb
The installation process also requires linux-header files.

@EnziinSystem, I updated my original post, but thought I'd add the reply here, too. I was running in a user context, but when I ran it with sudo, it worked perfectly. It's properly running on my Frigate NVR now.

hillzzz commented 1 month ago

@feranick

Today 05/13/2024, TensorFlow Lite released 2.17 and I can build it successfully on Pi-5.
pip3 install pybind11

git clone https://github.com/tensorflow/tensorflow.git

PYTHON=python3 tensorflow/tensorflow/lite/tools/pip_package/build_pip_package_with_cmake.sh native
Since you mentioned this before, what improvements or problems does tflite-2.17 have for pycoral?

I can confirm a successful build but I followed @feranick 's very helpful instructions and the build no from RELEASE.md is #

Release 2.18.0

TensorFlow

I am running the Dual ai Pineberry hat with the Dual EdgeTPU card. Python 3.11.2, Kernel Linux Raspian 6.6.32-v8+ #1769 SMP PREEMPT Thu Jun 6 11:38:54 BST 2024 aarch64 GNU/Linux

feranick commented 1 month ago

I'd note, once again, that the current stable version of TF (standard and lite) is 2.16.1. You may see more recent versions compiled IF, and only IF you compile from the master head of the git repo, which also means that you are running a very unstable version. If you want to run stable, you should get the latest stable version from here:

https://github.com/tensorflow/tensorflow/releases

Note: Google usually releases "RC" versions of the next stable in the same page, for testing. As for myself, I may do that but ultimately only publish builds against stable.

feranick commented 3 weeks ago

New test builds for the recently released TF 2.17.0-rc0. Note this is a release candidate, not stable, yet. Builds for tflite_runtime: https://github.com/feranick/TFlite-builds/releases/tag/v2.17.0-rc0 Builds for libedgetpu: https://github.com/feranick/libedgetpu/releases/tag/16.0TF2.17.0rc0-1 Builds for pycoral: https://github.com/feranick/pycoral/releases/tag/v2.0.2TF2.17.0rc0

Note: I have not fully test them, so use at your own risk. Note 2: If you don't see your platform, it means that at this stage iit is not supported.

Alvarofg commented 4 hours ago

TensorFlow 2.17.0 is out now. I hope we see some activity here soon :)

feranick commented 18 minutes ago

New builds for the recently released TF 2.17.0. Builds for tflite_runtime: https://github.com/feranick/TFlite-builds/releases/tag/v2.17.0 Builds for libedgetpu: https://github.com/feranick/libedgetpu/releases/tag/16.0TF2.17.0-1 Builds for pycoral: https://github.com/feranick/pycoral/releases/tag/2.0.2TF2.17.0

Note: If you don't see your platform, it means that at this stage iit is not supported.

An initial PR has been submitted to merge my changes to the official libesgetpu: https://github.com/google-coral/libedgetpu/pull/70

Once (if?) htat is merged, another one will be sent out for libcoral/pycoral.

google-coral / pycoral

Update pycoral for Python 3.11 and modern versions of Tensorflow #137

Description

TensorFlow