Stypox / dicio-android

Dicio assistant app for Android
GNU General Public License v3.0
726 stars 69 forks source link

Language translation skill #10

Open pixincreate opened 2 years ago

pixincreate commented 2 years ago

Lingva.ml is a open source front-end of Google Translate that contains no trackers of Google or any kind of things.
It is already being deployed and being continuously updated.
It would be great to see if you implement that as a webview part or any kind of thing in Dicio-Assistant.

Ask me anything.

I type translate "something" to .
And it throws an output that is done in lingva.ml.

And, App opener is really annoying and buggy. It really pisses me off when ever I ask it to open an app, it opens something other.
Ex: I activate Dicio through edge swipe action as I cannot start it with a voice. I ask it to 'open dialer' and it opens 'camera'! Yes, dialer app's name is 'Phone'. And I find it hard to say, open Phone as I'm already using the phone.

It's really helpful if you make it recognise all the keywords.
Or tell me what stuffs needs to be learnt, I'll learn and try to implement that, and send a pull request.:D

Stypox commented 2 years ago

Lingva.ml is a open source front-end of Google Translate that contains no trackers of Google or any kind of things.

Oh, great! I took a look at the API: it is really simple to use, it does not require any authentication and it also provides audio results. For reference, this is the repo: https://github.com/TheDavidDelta/lingva-translate

And, App opener is really annoying and buggy.

Yeah, it's not the best. When you say "Open something", that "something" is compared to the name of all apps on the phone and if there is an app whose name has a Levenshtein string distance smaller than or equal to 5 it is chosen. It works well if you say e.g. "Open NewPipe", but obviously there is not a unique name for the dialer. Maybe a map of synonyms should be created and searched through each time? E.g. it would look like [["dialer", "phone", "telephone", ...], ["maps", "magic earth", "osmand", "google maps", ...], ["youtube", "newpipe", ...], ["photos", "gallery", ...]], so that if you say "Open maps" the app "OsmAnd" is installed on your phone, that one is opened. What do you think?

Or tell me what stuffs needs to be learnt, I'll learn and try to implement that, and send a pull request.:D

The code that chooses the app to open is here: https://github.com/Stypox/dicio-android/blob/a0cb72f5084b4066c6631e6a84f487d86ef18688/app/src/main/java/org/dicio/dicio_android/skills/open/OpenOutput.java#L54 You shouldn't need to edit anything besides that one file. I think you should introduce a private static final List<Set<String>> variable with the contents described above, and then you can improve the getMostSimilarApp function. Thank you :-)

pixincreate commented 2 years ago

...that if you say "Open maps" the app "OsmAnd" is installed on your phone, that one is opened. What do you think?

That's looks and feels cool tho, have to try... I need to learn some Java now. Coming from C++(90s version, learning to work with the 20s version), it should not take much time IMO.

Stypox commented 2 years ago

Thank you! If you have any question or need some help feel free to ask :-)

Tombstone2K commented 2 years ago

. it would look like [["dialer", "phone", "telephone", ...], ["maps", "magic earth", "osmand", "google maps", ...], ["youtube", "newpipe", ...], ["photos", "gallery", ...]], so that if you say "Open maps" the app "OsmAnd" is installed on your phone, that one is opened. What do you think?

Yeah this seems really good

Shura0 commented 2 years ago

E.g. it would look like [["dialer", "phone", "telephone", ...], ["maps", "magic earth", "osmand", "google maps", ...], ["youtube", "newpipe", ...], ["photos", "gallery", ...]], so that if you say "Open maps" the app "OsmAnd" is installed on your phone, that one is opened. What do you think?

Sounds great. Also, maybe it should be a dialogue form?

open maps

< I found two application related to maps, is:

  1. Google maps
  2. OSMAnd Which one do you want to run?

    two

/dicio running osmand and saves the choice/

Tombstone2K commented 2 years ago

Maybe a menu in the settings that can choose which app to open when you say open maps....giving users the choice is good

ghost commented 2 years ago

Full Disclosure, I'm the maintainer of the SimplyTranslate project (https://simplytranslate.org) another open source and anonymous frontend for Google Translate and a couple other Translator as well, the main reason why you may want to consider it as well, is that we provide a list of instances here: https://simple-web.org/instances/simplytranslate which is constantly updated, also, the API may be simpler, and it also supports TTS which routes the audio through our servers which is more anonymous than Lingva's solution. Just putting it out there :)

pixincreate commented 2 years ago

I think you should introduce a private static final List<Set> variable with the contents described above,

Can I get some more details about this @Stypox if you don't mind? I'm finding it difficult to figure out where am I going wrong as I've almost 0 experience with Java for Android. created a list of set that needs to be appended with the words as described previously.

Stypox commented 2 years ago

This is an idea of how you could proceed, given you have private static final List<Set<String>> appGroups = /* ... */;, in pseudocode:

function getMostSimilarApp(packageManager, appName):
    # find `bestApplicationInfo` like done currently (i.e. using `customStringDistance` (!))
    # also save the `bestApplicationInfoScore` (currently `bestDistance`)

    bestAppGroup = null
    bestAppGroupScore = Integer.MAX_VALUE
    for each group in appGroups:
        for each str in group:
            str's score = customStringDistance(appName, str) (!)
            if str's score is lower than bestAppGroupScore:
                bestAppGroup = group
                bestAppGroupScore = str's score
    # now bestAppGroup contains the group containing an app name most similar to what the user said

    bestAppGroupInfo = null
    bestAppGroupInfoScore = Integer.MAX_VALUE
    for each str in bestAppGroup:
        for each resolveInfo in resolveInfos (like above):
            resolveInfo's score = levenshteinDistance(str, resolveInfo's app name (like above)) (!)
            if resolveInfo's score is lower than bestAppGroupInfoScore:
                bestAppGroupInfo = resolveInfo
                bestAppGroupInfoScore = resolveInfo's score
    # now bestAppGroupInfo contains the most similar app found on the device that matches one of the strings in the group

    # choose whether to return `bestApplicationInfo`, `bestAppGroupInfo` or `null` based on `bestApplicationInfoScore`,
    # `bestAppGroupScore` and `bestAppGroupInfoScore`; I'll let you experiment with the numbers to find a good heuristic

The places where I put (!) signal a different string distance function being used. I think it makes sense to compare the user input with any other string (i.e. on-device or inside-group app name) using customStringDistance, but to use levenshteinDistance when comparing the hardcoded inside-group app names with the on-device app names at the end. That's because the user might have pronounced the word differently than expected, and in that case customStringDistance works better, but when comparing exact strings levenshteinDistance seems more appropriate.

pixincreate commented 2 years ago

Ok, thank you for the valuable into.

I need some more time as I cannot focus on anything much now other then college projects. Will contribute in the coming days <3

Stypox commented 2 years ago

See #81 for the discussion about app synonyms

Stypox commented 1 year ago

From #127:

It would be nice if we could dictate something to translate to dicio He would use a service like LibreTranslate or even lingva