Explore using the on-device LLM for various tasks - Githubissues

openfoodfacts / smooth-app

The new Open Food Facts mobile application for Android and iOS, crafted with Flutter and Dart

https://world.openfoodfacts.org/open-food-facts-mobile-app?utm_source=off&utf_medium=web&utm_campaign=github-repo

Apache License 2.0

748 stars 261 forks source link

Explore using the on-device LLM for various tasks #5009

Open teolemon opened 5 months ago

teolemon commented 5 months ago

What

Explore using on-device LLMs for various tasks. To be clear, this is highly speculative, since only Android has much of the above, and only recent high-end Android have an on-device LLM.

Possible use-cases

### Possible use-cases
- [ ] Food intake journaling
- [ ] Ingredient analysis
- [ ] FAQ/Support (chat?)
- [ ] other use-cases TBD
- [ ] https://github.com/openfoodfacts/openfoodfacts-ai/issues/341

Caveats

May drain battery quickly depending on use-cases
Will need to be limited to recent devices (Pixel 6 and higher, at the current date - January 2024). This might change in the future.

How

Fully open on-device LLM: https://ai.google/build/project-guides/llm-on-android-with-keras-and-tensorflow-lite
https://android-developers.googleblog.com/2023/12/leverage-generative-ai-in-your-android-apps.html
App with LLM template: https://developer.android.com/studio/preview/gemini-template
Server side LLMs using a SDK: https://ai.google.dev/tutorials/android_quickstart
On-device Gemini Nano: https://ai.google.dev/tutorials/android_aicore?hl=fr

Originally posted by @teolemon in https://github.com/openfoodfacts/smooth-app/issues/4027#issuecomment-1907898318

Part of

4027

teolemon commented 5 months ago

@AshAman999 I clarified the situation