search

synesthesiam / rhasspy

Rhasspy voice assistant for offline home automation
https://rhasspy.readthedocs.io
MIT License
944 stars 101 forks source link

Missing words from ASR in raw_text #120

Closed mathquis closed 4 years ago

mathquis commented 4 years ago

I sometimes get missing words at the beginning or the end of the raw_text property.

The ASR decoding is correct.

Ex: Intent: allume (la lumière){what} dans (le salon){where} ASR: allume la lumière dans le salon raw_text: la lumière dans le salon

Looks like a bug.

Here is an extract from the logs:

DEBUG:KaldiDecoder:['bash', '/profiles/fr/kaldi/model/decode.sh', '/opt/kaldi', '/profiles/fr/kaldi/model', '/profiles/fr/kaldi/model/graph', '/tmp/tmpyau8q_gf.wav']
DEBUG:KaldiDecoder:allume la lumière de la salle à manger
DEBUG:__main__:allume la lumière de la salle à manger
DEBUG:__main__:{"intent": {"name": "onSwitchTurnOn", "confidence": 0.8571428571428572}, "entities": [{"entity": "type", "value": "light", "raw_value": "la lumi\u00e8re", "start": 0, "raw_start": 0, "end": 5, "raw_end": 10, "tokens": ["light"], "raw_tokens": ["la", "lumi\u00e8re"]}, {"entity": "zone", "value": "group.dining_room", "raw_value": "de la salle \u00e0 manger", "start": 6, "raw_start": 11, "end": 23, "raw_end": 31, "tokens": ["group.dining_room"], "raw_tokens": ["de", "la", "salle", "\u00e0", "manger"]}], "text": "light group.dining_room", "raw_text": "la lumi\u00e8re de la salle \u00e0 manger", "recognize_seconds": 0.11763829999836162, "tokens": ["light", "group.dining_room"], "raw_tokens": ["la", "lumi\u00e8re", "de", "la", "salle", "\u00e0", "manger"], "speech_confidence": 1, "slots": {"type": "light", "zone": "group.dining_room"}, "wakeId": "", "siteId": "default", "time_sec": 1.0456862449645996}
synesthesiam commented 4 years ago

I may need the complete intent with all sentences to debug this one. I have no idea off the top of my head...

mathquis commented 4 years ago

I have many intents and they are quite complex. I think this happens inside the NLU part. It is splitting the text as tokens and reconstructing the text and raw text using the graph. It looks like some tokens are missing afterwards. What files would you need? intents.json? intents.fst?

synesthesiam commented 4 years ago

I'm hoping this is fixed now with rhasspynlu version 0.1.4. If you're on Docker, I'll be pushing an image later in the day.

synesthesiam commented 4 years ago

Is this still happening?