mysamai / mysam

An open "intelligent" assistant for the web that can listen to you and learn.
https://mysamai.com
GNU General Public License v3.0
621 stars 65 forks source link

Missing translation/voice recogniton/ability to choose language #13

Open FliiFe opened 8 years ago

FliiFe commented 8 years ago

Hi.

I noticed the app understands the language your computer is in. For example, if I am on a chinese computer, I will not be able to speak english if I am not chinese. However, this is a good thing if you speak the language your computer understands.

It's not annoying once everything is reconfigured, but maybe adding some languages templates would be great.

igorantun commented 8 years ago

Instead of having "language templates" or specific strings for specific languages, we could use some module to translate the result of the recognition, and compare it to the english equivalent string?

Example: The computer language is detected as Portuguese, therefore the recognition API is set to recognize pt-BR speech. User says "Meu nome é Igor", which then gets translated to "My name is Igor" and analyzed.

tl;dr: Use a module to translate the speech to en-US text

FliiFe commented 8 years ago

Yeah, that's a better idea, as it does not require to make an exauhstive list of supported languages. If such module can be used, it can be used to traduce output too.

Example (French talking to MySam): User : "Je m'appelle John" Software translates : "My name is John" Software analyses... Pre-Output : "Hi John." Final output: "Bonjour John."

daffl commented 8 years ago

The problem is that the tagging mechanism when training something new won't work using translations. It also might be fairly slow.

In theory the classification should still work with other languages, there just aren't any non-english training sets yet. I can give it a shot with German and see how well it does.

There definitely should be an option to change your input language.

igorantun commented 8 years ago

True that. Well, it could use both. Training sets/language templates if avaiblable, and use translation as a fallback.

Yes, there should be an option to change the input language.

FliiFe commented 8 years ago

@daffl Why wouldn't tagging work using translation ? Mysam receives input in english, and outputs english, with some kind of wrapper that do the translation

(I'm not saying it's false, I just don't understand)

daffl commented 8 years ago

Well let's say you have a German input like

Ich heisse David

With the name tag at position 2 but it would translate to

My name is David

Now the name tag is at position 3. There is still a lot of room for improvement for a smarter the tagging mechanism but like I mentioned I think everything may already work with other languages.

FliiFe commented 8 years ago

But this is if MySam takes the original input in account. If it gets translated, and _then_ mysam analyses, It will just receive

My name is David

Won't it ?

daffl commented 8 years ago

But a user has to tag their own input when teaching (see https://youtu.be/VxFtSsCM_bo?t=1m46s). So the only way with using a translation would be to select it from the translated text.

igorantun commented 8 years ago

True that, makes sense.