search

commons-app / apps-android-commons

The Wikimedia Commons Android app allows users to upload pictures from their Android phone/tablet to Wikimedia Commons
https://commons-app.github.io/
Apache License 2.0
982 stars 1.18k forks source link

Detect wrongly-set caption language (LLM?) #5709

Open nicolas-raoul opened 2 months ago

nicolas-raoul commented 2 months ago

Many Commons contributors contribute in various languages, for instance in Urdu when posting a picture of a local dish then in English when posting a picture showing a technology. That's great, but they often forget to select the right language for the caption:

Screenshot_20240502-095800.png

The app should try to detect when there seems to be a language mismatch, and show a popup such as:

Your caption seems to be in Japanese, but English is declared as the caption language. Do you want to declare the caption language as being Japanese?

Implementation: Per our privacy policy we can not call third-party APIs. This task is probably not important enough to justify specific ML/LLM model training, but it is a great use case for device-embedded LLM on devices where that feature is available.

shankarpriyank commented 2 months ago

The idea is great, but I dont know about any device embedded LLMs @nicolas-raoul can you point me to some LLMs that you may know about?

nicolas-raoul commented 2 months ago

@shankarpriyank I think Pixel 8+ and recent Samsung devices are planned to get that soon. Hopefully the emulator will get it soon too.

https://developer.android.com/ai/aicore

https://blog.google/products/pixel/pixel-feature-drop-december-2023/

https://www.samsung.com/us/galaxy-ai/

karyotakisg commented 2 months ago

We consider working on it with @vtalos. Maybe tools like Apache Tika can be suitable for language recognition. Αlternatively, there are Github repositories that offer similar functionality, like https://github.com/shuyo/language-detection or https://github.com/optimaize/language-detector. What do you think? @nicolas-raoul

nicolas-raoul commented 2 months ago

@karyotakisg These projects would add weight to the APK, and as I said it is low-priority. Also, embedded LLM sounds like a fun thing to try. :-) If anyone has a device with AiCore (I think that currently means Samsung S23 Ultra or Pixel 8 Pro) please let us know.