search

falabrasil / kaldi-br

☕🇧🇷 Scripts para o Kaldi em Português Brasileiro
MIT License
46 stars 7 forks source link

Is it possible to add Kaldi-br into Rhasspy voice assistant? #13

Closed ArlindoFNeto closed 6 months ago

ArlindoFNeto commented 1 year ago

Hi guys!

First of all, I would like to congratulate you for the relevant work!

I am a user of Rhasspy, a beautiful offline voice assistant tool that has an easy, but extremely flexible, interface for voice-activated actions implementation, and it is open source, of course! Currently, Rhasspy supports Portuguese, from Portugal if I'm not mistaken, through Pocketsphinx, but its accuracy is poor. On the other hand, it works very well with Kaldi in English.

Considering technical and legal aspects, would it be possible to train a model to be inserted into Rhasspy? Having Kaldi's efficiency combined with training in our Portuguese language would make Rhasspy perfect for use in Brazil.

Kind regards!

cassiotbatista commented 1 year ago

Hi,

Thanks for reaching out.

I think it's possible, but not sure if it is that straightforward. Have you tried any of the models available for Vosk? Our model is on the list alongside another model for Portuguese: https://alphacephei.com/vosk/models

I'm not familiar with Rhasspy's ecossystem but I believe that with some tweaks it may even work out of the box* if you put the right files at the right place. IIUC you'll need a Kaldi-compliant nnet3 acoustic model (AM), an n-gram-based language model (LM) in ARPA format, and probably some phonetic dictionary. I'll provide you some links below (which are basically a separate version of the files that go into FalaBrasil's model for Vosk.)

* probably won't work for unknown words, tho. You gotta make sure all words you need for your assistant to work are in the dictionary, otherwise you'd need to add them at inference time, which I'm not sure Rhasspy is able to do w.r.t. Kaldi models.

EDIT: let me know how it goes :)

ArlindoFNeto commented 1 year ago

Hello Cassio,

Thank you for the quick response.

That's a great news! I'll create an environment to combine all the files you indicated and test it with Rhasspy ASR Kaldi (https://github.com/rhasspy/rhasspy-asr-kaldi). I'll keep you in touch for any good news... or troubles. :smile: