google-ai-edge / LiteRT

LiteRT is the new name for TensorFlow Lite (TFLite). While the name is new, it's still the same trusted, high-performance runtime for on-device AI, now with an expanded vision.
https://ai.google.dev/edge/litert
Apache License 2.0
163 stars 13 forks source link

LiteRT runtime build not working for Linux+Aarch64 #5

Open mansnils opened 2 months ago

mansnils commented 2 months ago

There is no aarch64 linux wheel for ai-edge-litert. https://pypi.org/project/ai-edge-litert/#files

For tensorflow there is e.g. CPython 3.11 manylinux: glibc 2.17+ ARM64 https://pypi.org/project/tensorflow/#files

It appears not to be possible to build natively for aarch64+linux from source (using the ci/build_pip_package_with_bazel.sh script):

How to reproduce on Linux+Aarch64: ./ci/build_pip_package_with_bazel.sh

Building TFL runtime in tensorflow on Linux+Aarch64 works: ./tensorflow/lite/tools/pip_package/build_pip_package_with_cmake.sh

byarbrough commented 1 month ago

Bummed that I cannot install on the Pi 4 running Linux raspberrypi 6.6.51+rpt-rpi-v8 #1 SMP PREEMPT Debian 1:6.6.51-1+rpt3 (2024-10-08) aarch64 GNU/Linux

But the docs do still say to use

pip install tflite-runtime

https://ai.google.dev/edge/litert/microcontrollers/python

AllanKamimura commented 1 month ago

yeah..... it's really curious how a library aimed for edge-runtime don't have an arm64 build available I asked if they have a date planned for the release of the linux/arm64 build, but didn't get any answer

in the end, I build it myself using that ci script and it is working :upside_down_face: python 3.11 linux/arm64

ai_edge_litert_nightly-1.0.1.dev20241024-cp311-cp311-linux_aarch64.zip

well, use it at your own discretion :sweat_smile:

johnthagen commented 1 month ago

aarch64 wheels are needed to make ai-edge-litert a true replacement for tflite-runtime, which provides manylinux_2_34_aarch64.whl wheels.

Another example where this is needed is for macOS ARM64 local development within containers-- this requires an ARM64 Linux wheel when installing into the container.

AndreasX21 commented 41 minutes ago

Ran into the same issue, we need linux/arm64 wheels asap!