search

synesthesiam / rhasspy

Rhasspy voice assistant for offline home automation
https://rhasspy.readthedocs.io
MIT License
950 stars 101 forks source link

Swedish language not working #106

Closed solidssss closed 4 years ago

solidssss commented 4 years ago

Hi.

I have installed rhasspy via add-on in hassio. When i try to train the profile I get this error in the "Problems" tab:

KaldiDecoder | Missing online.conf | Configuration file not found at /share/rhasspy/profiles/sv/model/online/conf/online.conf. Did you train your profile?

There is no online.conf file in the folder.

I have set "profile" to "sv" in the configuration and redownloaded the profile multiple times but it still gives me error. I have also tried to installing this profile but it still doesn't work.

This is the log when I press "Train":

[INFO:39100] quart.serving: 192.168.1.224:44391 GET /api/problems 1.1 200 412 7028
[INFO:39042] quart.serving: 192.168.1.224:44391 POST /api/train 1.1 200 36 6457659
[DEBUG:39032] DialogueManager: ready -> asleep
[INFO:39030] DialogueManager: Automatically listening for wake word
[DEBUG:39027] DialogueManager: training_loading -> ready
[INFO:39026] DialogueManager: Actors reloaded
[DEBUG:39018] FsticuffsRecognizer: started -> loaded
[DEBUG:39011] FsticuffsRecognizer: Using stop words at /share/rhasspy/profiles/sv/stop_words.txt
[DEBUG:38985] FsticuffsRecognizer:  -> started
[DEBUG:38985] KaldiDecoder:  -> started
[DEBUG:38984] DummyWakeListener:  -> started
[DEBUG:38983] DialogueManager: training_intent -> training_loading
[DEBUG:38982] FsticuffsRecognizer: loaded -> stopped
[DEBUG:38979] KaldiDecoder: started -> stopped
[DEBUG:38976] DummyWakeListener: started -> stopped
[DEBUG:38974] DialogueManager: Reloading actors
[INFO:38973] DialogueManager: Training complete
[DEBUG:38967] DialogueManager: training_sentences -> training_intent
[DEBUG:32627] rhasspynlu.ini_jsgf: Loaded ini file
[DEBUG:32622] train: Parsing ini file(s): ['/share/rhasspy/profiles/sv/sentences.ini']
[INFO:32606] quart.serving: 192.168.1.224:44392 GET /api/unknown-words 1.1 200 2 20880
[DEBUG:32594] DialogueManager: asleep -> training_sentences
[INFO:32588] __main__: Starting training

When I restart the add-on I cannot choose Pocketsphinx, it says Not compatible with this profile. If I press "save settings" in the settings tab it generates a profile.json in the "sv" folder. When I reboot after that, Pocketsphinx is automatically chosen. If I press Train now, I get a different error:

PocketsphinxDecoder | Missing acoustic model | Acoustic model directory not found at profiles/sv/acoustic_model. Did you download your profile?

This is the log in this scenario:

[INFO:958344] quart.serving: 192.168.1.224:54959 GET /api/problems 1.1 200 425 8934
[ERROR:958340] PocketsphinxWakeListener: on_receive
Traceback (most recent call last):
  File "/usr/share/rhasspy/rhasspy/actor.py", line 169, in on_receive
    self._state_method(message, sender)
  File "/usr/share/rhasspy/rhasspy/wake.py", line 139, in in_loaded
    self.load_decoder()
  File "/usr/share/rhasspy/rhasspy/wake.py", line 276, in load_decoder
    self.decoder = pocketsphinx.Decoder(decoder_config)
  File "/usr/local/lib/python3.6/dist-packages/pocketsphinx/pocketsphinx.py", line 273, in __init__
    this = _pocketsphinx.new_Decoder(*args)
RuntimeError: new_Decoder returned -1
[INFO:958290] quart.serving: 192.168.1.224:54959 POST /api/train 1.1 200 36 310761
[DEBUG:958286] PocketsphinxWakeListener: Loading wake decoder with hmm=profiles/sv/acoustic_model, dict=/share/rhasspy/profiles/sv/dictionary.txt
[WARNING:958284] PocketsphinxWakeListener: rhasspy not in dictionary
[WARNING:958282] PocketsphinxWakeListener: okay not in dictionary
[DEBUG:958272] DialogueManager: ready -> asleep
[INFO:958272] DialogueManager: Automatically listening for wake word
[DEBUG:958270] DialogueManager: training_loading -> ready
[INFO:958270] DialogueManager: Actors reloaded
[DEBUG:958266] PocketsphinxDecoder: started -> loaded
[WARNING:958264] PocketsphinxDecoder: preload: new_Decoder returned -1
[DEBUG:958193] PocketsphinxDecoder: Loading decoder with hmm=profiles/sv/acoustic_model, dict=/share/rhasspy/profiles/sv/dictionary.txt, lm=/share/rhasspy/profiles/sv/language_model.txt
[DEBUG:958188] PocketsphinxWakeListener: started -> loaded
[ERROR:958187] PocketsphinxWakeListener: loading wake decoder
Traceback (most recent call last):
  File "/usr/share/rhasspy/rhasspy/wake.py", line 130, in to_started
    self.load_decoder()
  File "/usr/share/rhasspy/rhasspy/wake.py", line 276, in load_decoder
    self.decoder = pocketsphinx.Decoder(decoder_config)
  File "/usr/local/lib/python3.6/dist-packages/pocketsphinx/pocketsphinx.py", line 273, in __init__
    this = _pocketsphinx.new_Decoder(*args)
RuntimeError: new_Decoder returned -1
[DEBUG:958119] PocketsphinxWakeListener: Loading wake decoder with hmm=profiles/sv/acoustic_model, dict=/share/rhasspy/profiles/sv/dictionary.txt
[WARNING:958118] PocketsphinxWakeListener: rhasspy not in dictionary
[WARNING:958116] PocketsphinxWakeListener: okay not in dictionary
[DEBUG:958115] FuzzyWuzzyRecognizer: started -> loaded
[DEBUG:958113] FuzzyWuzzyRecognizer: Loaded examples from /share/rhasspy/profiles/sv/intent_examples.json
[DEBUG:958102] PocketsphinxDecoder:  -> started
[DEBUG:958099] FuzzyWuzzyRecognizer:  -> started
[DEBUG:958098] PocketsphinxWakeListener:  -> started
[DEBUG:958098] DialogueManager: training_intent -> training_loading
[DEBUG:958093] FuzzyWuzzyRecognizer: loaded -> stopped
[DEBUG:958091] PocketsphinxDecoder: loaded -> stopped
[DEBUG:958088] PocketsphinxWakeListener: loaded -> stopped
[DEBUG:958087] DialogueManager: Reloading actors
[INFO:958086] DialogueManager: Training complete
[DEBUG:958083] FuzzyWuzzyIntentTrainer: Wrote intent examples to /share/rhasspy/profiles/sv/intent_examples.json
[DEBUG:958045] DialogueManager: training_sentences -> training_intent
[DEBUG:958019] rhasspynlu.ini_jsgf: Loaded ini file
[DEBUG:958014] train: Parsing ini file(s): ['/share/rhasspy/profiles/sv/sentences.ini']
[WARNING:958007] train: Unsupported acoustic model type: pocketsphinx
[INFO:957999] quart.serving: 192.168.1.224:54960 GET /api/unknown-words 1.1 200 2 12569
[DEBUG:957990] DialogueManager: asleep -> training_sentences
[INFO:957984] __main__: Starting training

Anyone know what is causing this error?

synesthesiam commented 4 years ago

Something is off if Pocketsphinx is loading for Swedish. The Swedish profile only supports the Kaldi speech engine.

You can ignored the online.conf error (it'll disappear in the next update).

solidssss commented 4 years ago

Something is off if Pocketsphinx is loading for Swedish. The Swedish profile only supports the Kaldi speech engine.

You can ignored the online.conf error (it'll disappear in the next update).

Thanks for the help. I did not know that the Swedish profile only works with Kaldi. You're doing a great job with this, you're awesome!

synesthesiam commented 4 years ago

Thank you! The Swedish profile definitely needs some work. The only speech model I could find was not really intended for this purpose, so the performance may be poor. If you (or anyone else) happen to know of a different Pocketsphinx, Kaldi, or Julius speech (acoustic) model for Swedish, let me know.

solidssss commented 4 years ago

Thank you! The Swedish profile definitely needs some work. The only speech model I could find was not really intended for this purpose, so the performance may be poor. If you (or anyone else) happen to know of a different Pocketsphinx, Kaldi, or Julius speech (acoustic) model for Swedish, let me know.

I have not had the time test the speech recognition in rhasspy yet, so I don't know how it performs with swedish yet. I found OpenSLR with swedish and this website (julius) which has swedish acoustics (alternative download). You may already know about this, but I wanted to share what little I know.

solidssss commented 4 years ago

Thank you! The Swedish profile definitely needs some work. The only speech model I could find was not really intended for this purpose, so the performance may be poor. If you (or anyone else) happen to know of a different Pocketsphinx, Kaldi, or Julius speech (acoustic) model for Swedish, let me know.

I have tried to play around with the addon now, and most of the time it will fail to understand the words I'm saying. Some simple tasks works well, like turn off and on lights but other things like saying "start a timer" but in swedish wont work. I might be able to trim the words a little better in the "words" tab though.

Sadly, I can only do speech commands with "speech tab" and tap to record. If I use a wake word, the rhasspy addon crashes. I have tried both snowboy and porcupine, but when I say the wake word it makes the rhasspy addon crash. Here is the log:

DEBUG:DialogueManager:asleep -> awake
DEBUG:WebrtcvadCommandListener:Will timeout in 30 second(s)
DEBUG:WebrtcvadCommandListener:loaded -> listening
DEBUG:SnowboyWakeListener:listening -> loaded
DEBUG:APlayAudioPlayer:['aplay', '-q', '/usr/share/rhasspy/etc/wav/beep_hi.wav']
DEBUG:PyAudioRecorder:recording -> started
DEBUG:PyAudioRecorder:Stopped recording from microphone (PyAudio)
DEBUG:PyAudioRecorder:started -> recording
Expression 'alsa_snd_pcm_hw_params_set_period_size_near( pcm, hwParams, &alsaPeriodFrames, &dir )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 924
Expression 'alsa_snd_pcm_hw_params_set_period_size_near( pcm, hwParams, &alsaPeriodFrames, &dir )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 924
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition 'cards.bcm2835_alsa.pcm.front.0:CARD=0'
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM front
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition 'cards.bcm2835_alsa.pcm.surround51.0:CARD=0'
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM surround21
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition 'cards.bcm2835_alsa.pcm.surround51.0:CARD=0'
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM surround21
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition 'cards.bcm2835_alsa.pcm.surround40.0:CARD=0'
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM surround40
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition 'cards.bcm2835_alsa.pcm.surround51.0:CARD=0'
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM surround41
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition 'cards.bcm2835_alsa.pcm.surround51.0:CARD=0'
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM surround50
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition 'cards.bcm2835_alsa.pcm.surround51.0:CARD=0'
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM surround51
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition 'cards.bcm2835_alsa.pcm.surround71.0:CARD=0'
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM surround71
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition 'cards.bcm2835_alsa.pcm.iec958.0:CARD=0,AES0=4,AES1=130,AES2=0,AES3=2'
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM iec958
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition 'cards.bcm2835_alsa.pcm.iec958.0:CARD=0,AES0=4,AES1=130,AES2=0,AES3=2'
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM spdif
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition 'cards.bcm2835_alsa.pcm.iec958.0:CARD=0,AES0=4,AES1=130,AES2=0,AES3=2'
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM spdif
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
Expression 'alsa_snd_pcm_hw_params_set_period_size_near( pcm, hwParams, &alsaPeriodFrames, &dir )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 924
ALSA lib pcm_dmix.c:1052:(snd_pcm_dmix_open) unable to open slave
/run.sh: line 18:    14 Segmentation fault      (core dumped) python3 app.py --user-profiles "${RHASSPY_USER_DIR}" --profile "${RHASSPY_PROFILE}" "$@"

Also, the "Audio Recording" input test in settings tab always gives me "(error)" when i try to click the button. (I'm using a Playstation eye as mic btw) Here is the log from the addon when I press "test":

INFO:quart.serving:192.168.1.224:5213 GET /api/test-microphones 1.1 200 212 3255192
[2019-12-23 10:37:14,529] 192.168.1.224:5213 GET /api/test-microphones 1.1 200 212 3255192
Expression 'parameters->channelCount <= maxChans' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1514
Expression 'ValidateParameters( inputParameters, hostApi, StreamDirection_In )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 2818
Expression 'parameters->channelCount <= maxChans' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1514
Expression 'ValidateParameters( inputParameters, hostApi, StreamDirection_In )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 2818
Expression 'ret' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1736
Expression 'AlsaOpen( &alsaApi->baseHostApiRep, params, streamDir, &self->pcm )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1904
Expression 'PaAlsaStreamComponent_Initialize( &self->capture, alsaApi, inParams, StreamDirection_In, NULL != callback )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 2171
Expression 'PaAlsaStream_Initialize( stream, alsaHostApi, inputParameters, outputParameters, sampleRate, framesPerBuffer, callback, streamFlags, userData )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 2840
Expression 'ret' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1736
Expression 'AlsaOpen( &alsaApi->baseHostApiRep, params, streamDir, &self->pcm )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1904
Expression 'PaAlsaStreamComponent_Initialize( &self->capture, alsaApi, inParams, StreamDirection_In, NULL != callback )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 2171
Expression 'PaAlsaStream_Initialize( stream, alsaHostApi, inputParameters, outputParameters, sampleRate, framesPerBuffer, callback, streamFlags, userData )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 2840
Expression 'parameters->channelCount <= maxChans' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1514
Expression 'ValidateParameters( inputParameters, hostApi, StreamDirection_In )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 2818
Expression 'ret' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1736
Expression 'AlsaOpen( &alsaApi->baseHostApiRep, params, streamDir, &self->pcm )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1904
Expression 'PaAlsaStreamComponent_Initialize( &self->capture, alsaApi, inParams, StreamDirection_In, NULL != callback )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 2171
Expression 'PaAlsaStream_Initialize( stream, alsaHostApi, inputParameters, outputParameters, sampleRate, framesPerBuffer, callback, streamFlags, userData )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 2840
synesthesiam commented 4 years ago

Great, I'll check out the models you linked.

I've been seeing a lot of people have problems lately with PyAudio. Can you try switching to Arecord in the Settings and see if you still have problems?

solidssss commented 4 years ago

Great, I'll check out the models you linked.

I've been seeing a lot of people have problems lately with PyAudio. Can you try switching to Arecord in the Settings and see if you still have problems?

Awesome. I would be so happy if you could improve the swedish speech in rhasspy. If rhasspy gets better with swedish some day, I'll replace my google assistant to rhasspy for sure.

Snowboy wakeword is now working after I updated to the latest rhasspy in hassio. PyAudio is still throwing error in the settings tab, but it seems to work anyway since rhasspy can hear my wake word and speech commands. Arecord is showing error in the settings and is not picking up on my voice when I speech.

solidssss commented 4 years ago

I got it working a lot better with voice commands if I use the pre-built copy of Kaldi. If I use the standard "${KALDI_PREFIX}/kaldi" it hardly ever works. But with the pre-build installation it works pretty good now! It still fails sometimes, but it's much better performance now.