openfoodfacts / smooth-app

The new Open Food Facts mobile application for Android and iOS, crafted with Flutter and Dart
https://world.openfoodfacts.org/open-food-facts-mobile-app?utm_source=off&utf_medium=web&utm_campaign=github-repo
Apache License 2.0
753 stars 261 forks source link

Supercharge the app with artificial intelligence (Tracker) #4027

Open teolemon opened 1 year ago

teolemon commented 1 year ago

What

### Tasks
- [ ] https://github.com/openfoodfacts/smooth-app/issues/3279
- [ ] https://github.com/openfoodfacts/smooth-app/issues/3986
- [ ] https://github.com/openfoodfacts/smooth-app/issues/3985
- [ ] https://github.com/openfoodfacts/smooth-app/issues/4069
- [ ] https://github.com/openfoodfacts/smooth-app/issues/3326
- [ ] https://github.com/openfoodfacts/smooth-app/issues/3775
- [ ] https://github.com/openfoodfacts/smooth-app/issues/3278
- [ ] https://github.com/openfoodfacts/smooth-app/issues/3340
- [ ] https://github.com/openfoodfacts/smooth-app/issues/2457
- [ ] Add optional offline ingredient extraction using MLKIT (disabled on the F-Droid build)
- [ ] Add offline packaging extraction using MLKIT (disabled on the F-Droid build)
- [ ] Explore the feasability of bringing OCR analysis done by Robotoff offline
- [ ] Explore the feasability of bringing nutrition analysis done by Robotoff offline
- [ ] Explore the feasability of bringing category prediction done by Robotoff offline
- [ ] https://github.com/openfoodfacts/smooth-app/issues/5007
- [ ] Offer on-device translation of products using MLKit
- [ ] https://github.com/openfoodfacts/smooth-app/issues/5006
- [ ] https://github.com/openfoodfacts/smooth-app/issues/5008
- [ ] https://github.com/openfoodfacts/smooth-app/issues/5401
- [ ] https://github.com/openfoodfacts/smooth-app/issues/5402
- [ ] Add an optional button on the product page to jump to Hunger Games crops and logos view for the product
- [ ] Use on-device classification to detect the type of product (Food, Cosmetic, PetFood, Products, or more granular) and show a warning
- [ ] https://github.com/openfoodfacts/smooth-app/issues/5387
- [ ] https://github.com/openfoodfacts/smooth-app/issues/5322

Part of

teolemon commented 11 months ago

@raphael0202 I added the idea to do on-device language recognition here

teolemon commented 6 months ago

What

Create a custom Tensorflow model for MLKit to classify Nutrition facts, Packaging instruction, Front and Ingredients photo on device Create a custom Tensorflow model for MLKit to classify food, pet food, cosmetics and other products on device

How

https://developers.google.com/ml-kit/vision/image-labeling?hl=fr https://developers.google.com/ml-kit/vision/face-detection?hl=fr

teolemon commented 6 months ago

Explore using the on-device LLM for various tasks