tingyang01
commented
4 years ago Hello, ldeseynes, I am wonderful that you get some considerable result for french stt. I've also tried to use gst-kaldi-nnet2-online to decode french speech. but I've got only one word for a call. So I've tested it on offline-decode, then it gives considerable result. I configured worker.yaml like this:
use-nnet2: True decoder:
All the properties nested here correspond to the kaldinnet2onlinedecoder GStreamer plugin properties.
# Use gst-inspect-1.0 ./libgstkaldionline2.so kaldinnet2onlinedecoder to discover the available properties
use-threaded-decoder: True
model : test/models/french/librifrench/final.mdl
word-syms : test/models/french/librifrench/words.txt
fst : test/models/french/librifrench/HCLG.fst
mfcc-config : test/models/french/librifrench/conf/mfcc.conf
ivector-extraction-config : test/models/french/librifrench/conf/ivector_extractor.conf
max-active: 10000
beam: 40.0
lattice-beam: 6.0
acoustic-scale: 0.083
do-endpointing : true
endpoint-silence-phones : "1:2:3:4:5:6:7:8:9:10"
traceback-period-in-secs: 0.25
chunk-length-in-secs: 0.25
num-nbest: 10
#Additional functionality that you can play with:
#lm-fst: test/models/english/librispeech_nnet_a_online/G.fst
#big-lm-const-arpa: test/models/english/librispeech_nnet_a_online/G.carpa
#phone-syms: test/models/english/librispeech_nnet_a_online/phones.txt
#word-boundary-file: test/models/english/librispeech_nnet_a_online/word_boundary.int
#do-phone-alignment: trueout-dir: tmp
use-vad: False silence-timeout: 10
Just a sample post-processor that appends "." to the hypothesis
post-processor: perl -npe 'BEGIN {use IO::Handle; STDOUT->autoflush(1);} s/(.*)/\1./;' Then I've got following log message.
-
gst worker server: 2019-09-18 01:37:14 - INFO: main: d01660ed-3b16-4401-a941-187b3bceb971: Postprocessing done. 2019-09-18 01:37:14 - DEBUG: main: d01660ed-3b16-4401-a941-187b3bceb971: After postprocessing: {u'status': 0, u'segment-start': 5.18, u'segment-length': 3.54, u'total-length': 8.72, u'result': {u'hypotheses': [{u'likelihood': -6.27475, u'transcript': u'il.', 'original-transcript': u'il'}, {u'likelihood': -7.54526, u'transcript': u'ils.', 'original-transcript': u'ils'}, {u'likelihood': -8.89724, u'transcript': u'il jeta.', 'original-transcript': u'il jeta'}, {u'likelihood': -10.2568, u'transcript': u'il je.', 'original-transcript': u'il je'}, {u'likelihood': -10.3752, u'transcript': u"il j'.", 'original-transcript': u"il j'"}, {u'likelihood': -10.9077, u'transcript': u'il ne.', 'original-transcript': u'il ne'}, {u'likelihood': -11.0849, u'transcript': u'ils je.', 'original-transcript': u'ils je'}, {u'likelihood': -11.1516, u'transcript': u"ils j'.", 'original-transcript': u"ils j'"}, {u'likelihood': -11.2717, u'transcript': u'il me.', 'original-transcript': u'il me'}, {u'likelihood': -11.2749, u'transcript': u'de.', 'original-transcript': u'de'}], u'final': True}, 'segment': 0, 'id': u'd01660ed-3b16-4401-a941-187b3bceb971'}
-
gst master server INFO 2019-09-18 01:37:09,763 d01660ed-3b16-4401-a941-187b3bceb971: Sending event {u'status': 0, u'segment': 0, u'result': {u'hypotheses': [{u'transcript': u'de.'}], u'final': Fal... to client INFO 2019-09-18 01:37:14,013 d01660ed-3b16-4401-a941-187b3bceb971: Sending event {u'status': 0, u'segment-start': 5.18, u'segment-length': 3.54, u'total-length': 8.72, u'result':... to client INFO 2019-09-18 01:37:14,024 d01660ed-3b16-4401-a941-187b3bceb971: Sending event {u'status': 0, u'adaptation_state': {u'type': u'string+gzip+base64', u'id': u'd01660ed-3b16-4401-... to client
-
client Audio sent, now sending EOS il.
I am very grateful what's wrong on my configuration. How can I get correct result? Thanks in helpness.
ldeseynes
Hi Tanel ! First of all thanks for your great job.
I'm using gst-kaldi-nnet2-online to decode french speech. When running the client.py script, I get a quite correct transcription using my model but at some point the decoding stops and starts again a few seconds later. This results in missing words in the output. Here is an example of the result with two dots corresponding to the missing words at the end of each sentence:
`bonjour , je m' appelle Jean-Christophe je suis agriculteur dans le Loiret sur une exploitation céréalières . je me suis installé il y a une dizaine d' années ..