search

google-ai-edge / LiteRT

LiteRT is the new name for TensorFlow Lite (TFLite). While the name is new, it's still the same trusted, high-performance runtime for on-device AI, now with an expanded vision.
https://ai.google.dev/edge/litert
Apache License 2.0
167 stars 13 forks source link

Slower inference since TensorFlow Lite 2.12 #91

Open gaikwadrahul8 opened 1 day ago

gaikwadrahul8 commented 1 day ago

System information

Standalone code to reproduce the issue I used the TensorFlow Lite Pose Estimation Android Demo where I changed the TensorFlow Lite version

Any other info / logs With any bundled version of TFLite >= 2.12.0, the inference time of some model is twice the inference time with TFLite v2.11.0. This issue is not reproduced with the TFLite from the Play Services.

Especially when I compared last week the bundled 2.13.0 against the 2.13.0 from the Play Services. The bundled version inference times were twice the ones of the Play Services. (I can't test it anymore since the version from the Play Services has been updated to 2.15.0 and this one is not available on Maven)

For example, on a Samsung Galaxy S23 ultra, the PoseNet model from the example app:

TFLite version: 2.11.0 (bundled) Average inference time: 12.28ms

TFLite version: 2.12.0 (bundled) Average inference time: 25.93ms

TFLite version: 2.13.0 (bundled) Average inference time: 26.03ms

TFLite version: 2.14.0 (bundled) Average inference time: 25.86ms

TFLite version: 2.15.0 (Google Play Services) Average inference time: 12.12ms

Any idea where that could come from ?

gaikwadrahul8 commented 9 hours ago

This issue originally reported by @Qheb has been moved to this dedicated repository for LiteRT to enhance issue tracking and prioritization. To ensure continuity, we have created this new issue on your behalf.

We appreciate your understanding and look forward to your continued involvement.