search

kalliope-project / kalliope

Kalliope is a framework that will help you to create your own personal assistant.
https://kalliope-project.github.io/
GNU General Public License v3.0
1.71k stars 230 forks source link

STT fallback #567

Open JulienSambre opened 4 years ago

JulienSambre commented 4 years ago

Hi!

I'm working heavily on Kalliope to make a lot of things and one of key feature is the offline STT (which is coherent with the purpose of having a self-hosted AI).

I'm starting using CMUSphinx wich will be fine I expect (just did the install + conf, will see the quality of matching during a few days).

I was asking myself how to handle unmatched orders, due to wrong voice analyze from CMUSphinx (which should be less effective than popular online STT, like Google).

How would it be possible to make a fallback to use another STT if an order is not matched? I already have a neurone to handle some case of unmatched order (i'm starting to work on a "machine learning" code to learn from unmatched orders), and it would be nice to call another STT when local fix of fail order is not possible. From python code for example or something else.

Sispheor commented 4 years ago

Technically it would be possible. But, from my point of view, a such feature would take time to process. And so the final result would not looks fluid enough deserve an implementation.

JulienSambre commented 4 years ago

I am curious to know how would you do that, technically. I have no clear idea (the documentation didn't give me an obvious idea) and I think I could go in an awful and complex solution that I have currently no idea :).

For the time to process, it is the key question. From my observations, orders are answered pretty fast (from my point of view) and adding a second a less to order process, only if an order is not match, would be acceptable. If you have an idea of how to do that, I would be glad to make the test and see how it works ;).

Sispheor commented 4 years ago

When I said it's technically possible I meant that it would be possible to implement it. So far it's not possible.

corus87 commented 4 years ago

I just opened a new issue about an order fallback, maybe @JulienSambre you get an idea how to implement an STT fallback too.

JulienSambre commented 4 years ago

@corus87 I gave it my 2 cents ;).

@Sispheor I tried for an hour to use CMUSphinx : I was unable to match a single order, this STT barely understand correctly a single word, while Google STT is nearly at 100%. For now, I'm stuck with online STT only. So I put my asking in pause, until I could make CMUSphinx understand what I say, even a little :/.

Sispheor commented 4 years ago

I leave the issue opened. Will see later if it's a must have feature. So far we do have a lot of refactoring in progress.