Closed cbsandeep10 closed 6 years ago
Did you change the paths configure in models/conf/ivector_extractor.conf file?
I checked the path of the files in ivector twice. Everything seems to be fine.
Did you download this model from somewhere? I look for it here (http://kaldi-asr.org/downloads/tree/trunk/egs/librispeech/s5/exp/) and here (https://api.ai/downloads/api.ai-kaldi-asr-model.zip), but I couldn't donwload it.
I would like to test here and find out the problem...
When I said Librispeech, I meant that I trained the model with our custom data using the below script https://github.com/kaldi-asr/kaldi/blob/master/egs/librispeech/s5/local/chain/run_tdnn.sh. I didn't download the official kaldi models. Is there any other parameters that I have to change related to this chain tdnn model.
I guess I need to use a different gst plugin for nnet3 chain models. I am currently using this plugin /opt/gst-kaldi-nnet2-online/src/libgstkaldionline2.so.
I've just tried here with a chain model and the docker image from docker hub. As you can see below, everything was fine. This chain model was trained in September 2017. I don't know when you trained your model and I don't know if there was any change in Kaldi file's format recently. Maybe, if there was such a change, the docker hub image can be outdated. So, you could try to build your own docker image. I don't know if it will solve your problem... it is just a guess.
It is my yaml file:
use-nnet2: True
decoder:
# All the properties nested here correspond to the kaldinnet2onlinedecoder GStreamer plugin properties.
# Use gst-inspect-1.0 ./libgstkaldionline2.so kaldinnet2onlinedecoder to discover the available properties
nnet-mode: 3
use-threaded-decoder: True
model : /opt/models/tdnn1c_rvb_lstm_sp_online/final.mdl
word-syms : /opt/models/words.txt
fst : /opt/models/HCLG.fst
mfcc-config : /opt/models/tdnn1c_rvb_lstm_sp_online/conf/mfcc.conf
ivector-extraction-config : /opt/models/tdnn1c_rvb_lstm_sp_online/conf/ivector_extractor.conf
frame-subsampling-factor: 3
#frames-per-chunk: 20
#extra-left-context-initial: 0
max-active: 10000
beam: 15.0
lattice-beam: 8.0
acoustic-scale: 1.25
# lmwt-scale: 0.8
do-endpointing : false
##endpoint-silence-phones : "1:2:3:4:5:6:7:8:9:10"
traceback-period-in-secs: 0.25
chunk-length-in-secs: 0.25
num-nbest: 10
#Additional functionality that you can play with:
# lm-fst: /opt/models/tdnn1c_rvb_lstm_sp_online/lang_small9_carpa/G.fst
# big-lm-const-arpa: /opt/models/tdnn1c_rvb_lstm_sp_online/lang_small9_carpa/G.carpa
# phone-syms: /opt/models/tdnn1c_rvb_lstm_sp_online/lang_small9_carpa/phones.txt
# word-boundary-file: /opt/models/tdnn1c_rvb_lstm_sp_online/lang_small9_carpa/phones/word_boundary.int
# do-phone-alignment: true
out-dir: /opt/tmp
use-vad: False
silence-timeout: 10
# Just a sample post-processor that appends "." to the hypothesis
post-processor: perl -npe 'BEGIN {use IO::Handle; STDOUT->autoflush(1);} s/(.*)/\1/;'
# A sample full post processor that add a confidence score to 1-best hyp and deletes other n-best hyps
#full-post-processor: /opt/kaldi-gstreamer-server/sample_full_post_processor.py
logging:
version : 1
disable_existing_loggers: False
formatters:
simpleFormater:
format: '%(asctime)s - %(levelname)7s: %(name)10s: %(message)s'
datefmt: '%Y-%m-%d %H:%M:%S'
handlers:
console:
class: logging.StreamHandler
formatter: simpleFormater
level: DEBUG
root:
level: DEBUG
handlers: [console]
and this was the output (worker.log):
root@a0750856fced:/opt# cat worker.log
libdc1394 error: Failed to initialize libdc1394
libudev: udev_has_devtmpfs: name_to_handle_at on /dev: Operation not permitted
DEBUG 2018-06-12 13:04:55,463 Starting up worker
2018-06-12 13:04:55 - INFO: decoder2: Creating decoder using conf: {'post-processor': "perl -npe 'BEGIN {use IO::Handle; STDOUT->autoflush(1);} s/(.*)/\\1/;'", 'logging': {'version': 1, 'root': {'level': 'DEBUG', 'handlers': ['console']}, 'formatters': {'simpleFormater': {'datefmt': '%Y-%m-%d %H:%M:%S', 'format': '%(asctime)s - %(levelname)7s: %(name)10s: %(message)s'}}, 'disable_existing_loggers': False, 'handlers': {'console': {'formatter': 'simpleFormater', 'class': 'logging.StreamHandler', 'level': 'DEBUG'}}}, 'use-vad': False, 'decoder': {'ivector-extraction-config': '/opt/models/tdnn1c_rvb_lstm_sp_online/conf/ivector_extractor.conf', 'lattice-beam': 8.0, 'acoustic-scale': 1.25, 'do-endpointing': False, 'beam': 15.0, 'mfcc-config': '/opt/models/tdnn1c_rvb_lstm_sp_online/conf/mfcc.conf', 'traceback-period-in-secs': 0.25, 'nnet-mode': 3, 'word-syms': '/opt/models/words.txt', 'num-nbest': 10, 'frame-subsampling-factor': 3, 'max-active': 10000, 'fst': '/opt/models/HCLG.fst', 'use-threaded-decoder': True, 'model': '/opt/models/tdnn1c_rvb_lstm_sp_online/final.mdl', 'chunk-length-in-secs': 0.25}, 'silence-timeout': 10, 'out-dir': '/opt/tmp', 'use-nnet2': True}
2018-06-12 13:04:55 - INFO: decoder2: Setting decoder property: nnet-mode = 3
2018-06-12 13:04:55 - INFO: decoder2: Setting decoder property: ivector-extraction-config = /opt/models/tdnn1c_rvb_lstm_sp_online/conf/ivector_extractor.conf
2018-06-12 13:04:55 - INFO: decoder2: Setting decoder property: lattice-beam = 8.0
2018-06-12 13:04:55 - INFO: decoder2: Setting decoder property: acoustic-scale = 1.25
2018-06-12 13:04:55 - INFO: decoder2: Setting decoder property: do-endpointing = False
2018-06-12 13:04:55 - INFO: decoder2: Setting decoder property: beam = 15.0
2018-06-12 13:04:55 - INFO: decoder2: Setting decoder property: mfcc-config = /opt/models/tdnn1c_rvb_lstm_sp_online/conf/mfcc.conf
2018-06-12 13:04:55 - INFO: decoder2: Setting decoder property: traceback-period-in-secs = 0.25
2018-06-12 13:04:55 - INFO: decoder2: Setting decoder property: word-syms = /opt/models/words.txt
2018-06-12 13:04:55 - INFO: decoder2: Setting decoder property: num-nbest = 10
2018-06-12 13:04:55 - INFO: decoder2: Setting decoder property: frame-subsampling-factor = 3
2018-06-12 13:04:55 - INFO: decoder2: Setting decoder property: max-active = 10000
2018-06-12 13:04:55 - INFO: decoder2: Setting decoder property: fst = /opt/models/HCLG.fst
2018-06-12 13:04:55 - INFO: decoder2: Setting decoder property: model = /opt/models/tdnn1c_rvb_lstm_sp_online/final.mdl
LOG ([5.2.187~1-148c8]:CompileLooped():nnet-compile-looped.cc:336) Spent 0.016602 seconds in looped compilation.
2018-06-12 13:04:56 - INFO: decoder2: Setting decoder property: chunk-length-in-secs = 0.25
2018-06-12 13:04:56 - INFO: decoder2: Created GStreamer elements
2018-06-12 13:04:56 - DEBUG: decoder2: Adding <__main__.GstAppSrc object at 0x7f2ac23ec3c0 (GstAppSrc at 0x21727b0)> to the pipeline
2018-06-12 13:04:56 - DEBUG: decoder2: Adding <__main__.GstDecodeBin object at 0x7f2ac23ec2d0 (GstDecodeBin at 0x21e4060)> to the pipeline
2018-06-12 13:04:56 - DEBUG: decoder2: Adding <__main__.GstAudioConvert object at 0x7f2ac23ec280 (GstAudioConvert at 0x21ee4e0)> to the pipeline
2018-06-12 13:04:56 - DEBUG: decoder2: Adding <__main__.GstAudioResample object at 0x7f2ac23ec370 (GstAudioResample at 0x1fb7430)> to the pipeline
2018-06-12 13:04:56 - DEBUG: decoder2: Adding <__main__.GstTee object at 0x7f2ac23ec230 (GstTee at 0x21fc000)> to the pipeline
2018-06-12 13:04:56 - DEBUG: decoder2: Adding <__main__.GstQueue object at 0x7f2ac23ec320 (GstQueue at 0x2200170)> to the pipeline
2018-06-12 13:04:56 - DEBUG: decoder2: Adding <__main__.GstFileSink object at 0x7f2ac23ec410 (GstFileSink at 0x2203200)> to the pipeline
2018-06-12 13:04:56 - DEBUG: decoder2: Adding <__main__.GstQueue object at 0x7f2ac23ec460 (GstQueue at 0x2200460)> to the pipeline
2018-06-12 13:04:56 - DEBUG: decoder2: Adding <__main__.Gstkaldinnet2onlinedecoder object at 0x7f2ac23ec4b0 (Gstkaldinnet2onlinedecoder at 0x221e150)> to the pipeline
2018-06-12 13:04:56 - DEBUG: decoder2: Adding <__main__.GstFakeSink object at 0x7f2ac23ec500 (GstFakeSink at 0x2038440)> to the pipeline
2018-06-12 13:04:56 - INFO: decoder2: Linking GStreamer elements
LOG ([5.2.187~1-148c8]:ComputeDerivedVars():ivector-extractor.cc:183) Computing derived variables for iVector extractor
LOG ([5.2.187~1-148c8]:ComputeDerivedVars():ivector-extractor.cc:204) Done.
2018-06-12 13:04:56 - INFO: decoder2: Setting pipeline to READY
2018-06-12 13:04:56 - INFO: decoder2: Set pipeline to READY
2018-06-12 13:04:56 - INFO: __main__: Opening websocket connection to master server
2018-06-12 13:04:56 - INFO: __main__: Opened websocket connection to server
Mine is a very recent one. I trained the model in April 2018.
I'm also using /opt/gst-kaldi-nnet2-online/src/libgstkaldionline2.so plugin. I don't believe you need to change it.
Some suggestions:
I think this way you have reasonable chances to solve your problem.
Thanks @jcsilva for helping me out.
Thank you so much @jcsilva . I built a new docker image and It worked perfectly.
I am using chain model from librispeech kaldi. I am getting this error while running start.sh.
root@0ca580342230:/opt# cat worker.log DEBUG 2018-06-12 06:35:24,111 Starting up worker 2018-06-12 06:35:24 - INFO: decoder2: Creating decoder using conf: {'post-processor': "perl -npe 'BEGIN {use IO::Handle; STDOUT->autoflush(1);} s/(.*)/\1./;'", 'logging': {'version': 1, 'root': {'level': 'DEBUG', 'handlers': ['console']}, 'formatters': {'simpleFormater': {'datefmt': '%Y-%m-%d %H:%M:%S', 'format': '%(asctime)s - %(levelname)7s: %(name)10s: %(message)s'}}, 'disable_existing_loggers': False, 'handlers': {'console': {'formatter': 'simpleFormater', 'class': 'logging.StreamHandler', 'level': 'DEBUG'}}}, 'use-vad': False, 'decoder': {'ivector-extraction-config': 'models/conf/ivector_extractor.conf', 'lattice-beam': 6.0, 'acoustic-scale': 1.0, 'do-endpointing': True, 'beam': 15.0, 'mfcc-config': 'models/conf/mfcc_hires.conf', 'traceback-period-in-secs': 0.25, 'nnet-mode': 3, 'endpoint-silence-phones': '1:2:3:4:5:6:7:8:9:10', 'word-syms': 'models/tdnn_sp/words.txt', 'num-nbest': 10, 'phone-syms': 'models/phones.txt', 'max-active': 7000, 'fst': 'models/tdnn_sp/HCLG.fst', 'use-threaded-decoder': True, 'model': 'models/tdnn_sp/final.mdl', 'chunk-length-in-secs': 0.25}, 'silence-timeout': 10, 'out-dir': 'tmp', 'use-nnet2': True} 2018-06-12 06:35:24 - INFO: decoder2: Setting decoder property: nnet-mode = 3 2018-06-12 06:35:24 - INFO: decoder2: Setting decoder property: ivector-extraction-config = models/conf/ivector_extractor.conf 2018-06-12 06:35:24 - INFO: decoder2: Setting decoder property: lattice-beam = 6.0 2018-06-12 06:35:24 - INFO: decoder2: Setting decoder property: acoustic-scale = 1.0 2018-06-12 06:35:24 - INFO: decoder2: Setting decoder property: do-endpointing = True 2018-06-12 06:35:24 - INFO: decoder2: Setting decoder property: beam = 15.0 2018-06-12 06:35:24 - INFO: decoder2: Setting decoder property: mfcc-config = models/conf/mfcc_hires.conf 2018-06-12 06:35:24 - INFO: decoder2: Setting decoder property: traceback-period-in-secs = 0.25 2018-06-12 06:35:24 - INFO: decoder2: Setting decoder property: endpoint-silence-phones = 1:2:3:4:5:6:7:8:9:10 2018-06-12 06:35:24 - INFO: decoder2: Setting decoder property: word-syms = models/tdnn_sp/words.txt 2018-06-12 06:35:24 - INFO: decoder2: Setting decoder property: num-nbest = 10 2018-06-12 06:35:24 - INFO: decoder2: Setting decoder property: phone-syms = models/phones.txt 2018-06-12 06:35:24 - INFO: decoder2: Setting decoder property: max-active = 7000 2018-06-12 06:35:24 - INFO: decoder2: Setting decoder property: fst = models/tdnn_sp/HCLG.fst 2018-06-12 06:35:25 - INFO: decoder2: Setting decoder property: model = models/tdnn_sp/final.mdl ERROR ([5.2.187~1-148c8]:Read():nnet-simple-component.cc:2463) Expected token, got
[ Stack-Trace: ] kaldi::MessageLogger::HandleMessage(kaldi::LogMessageEnvelope const&, char const*) kaldi::MessageLogger::~MessageLogger() kaldi::nnet3::NaturalGradientAffineComponent::Read(std::istream&, bool) kaldi::nnet3::Component::ReadNew(std::istream&, bool) kaldi::nnet3::Nnet::Read(std::istream&, bool) kaldi::nnet3::AmNnetSimple::Read(std::istream&, bool)
g_object_set_property
. . . python() [0x4b988b] PyEval_EvalFrameEx PyEval_EvalFrameEx PyEval_EvalCodeEx python() [0x50160f] PyRun_FileExFlags PyRun_SimpleFileExFlags Py_Main __libc_start_main python() [0x497b8b]
2018-06-12 06:35:25 - INFO: decoder2: Setting decoder property: chunk-length-in-secs = 0.25 2018-06-12 06:35:25 - INFO: decoder2: Created GStreamer elements 2018-06-12 06:35:25 - DEBUG: decoder2: Adding <main.GstAppSrc object at 0x7fdda14e03c0 (GstAppSrc at 0x2ec05b0)> to the pipeline 2018-06-12 06:35:25 - DEBUG: decoder2: Adding <main.GstDecodeBin object at 0x7fdda14e02d0 (GstDecodeBin at 0x2eb4080)> to the pipeline 2018-06-12 06:35:25 - DEBUG: decoder2: Adding <main.GstAudioConvert object at 0x7fdda14e0280 (GstAudioConvert at 0x2ed3a60)> to the pipeline 2018-06-12 06:35:25 - DEBUG: decoder2: Adding <main.GstAudioResample object at 0x7fdda14e0370 (GstAudioResample at 0x2d7c9f0)> to the pipeline 2018-06-12 06:35:25 - DEBUG: decoder2: Adding <main.GstTee object at 0x7fdda14e0230 (GstTee at 0x2ee4000)> to the pipeline 2018-06-12 06:35:25 - DEBUG: decoder2: Adding <main.GstQueue object at 0x7fdda14e0320 (GstQueue at 0x2ee8200)> to the pipeline 2018-06-12 06:35:25 - DEBUG: decoder2: Adding <main.GstFileSink object at 0x7fdda14e0410 (GstFileSink at 0x2eec020)> to the pipeline 2018-06-12 06:35:25 - DEBUG: decoder2: Adding <main.GstQueue object at 0x7fdda14e0460 (GstQueue at 0x2ee84f0)> to the pipeline 2018-06-12 06:35:25 - DEBUG: decoder2: Adding <main.Gstkaldinnet2onlinedecoder object at 0x7fdda14e04b0 (Gstkaldinnet2onlinedecoder at 0x2f0e090)> to the pipeline 2018-06-12 06:35:25 - DEBUG: decoder2: Adding <main.GstFakeSink object at 0x7fdda14e0500 (GstFakeSink at 0x2decfc0)> to the pipeline 2018-06-12 06:35:25 - INFO: decoder2: Linking GStreamer elements LOG ([5.2.187~1-148c8]:ComputeDerivedVars():ivector-extractor.cc:183) Computing derived variables for iVector extractor LOG ([5.2.187~1-148c8]:ComputeDerivedVars():ivector-extractor.cc:204) Done. 2018-06-12 06:35:25 - INFO: decoder2: Setting pipeline to READY 2018-06-12 06:35:25 - INFO: decoder2: Set pipeline to READY 2018-06-12 06:35:25 - INFO: main: Opening websocket connection to master server 2018-06-12 06:35:25 - INFO: main: Opened websocket connection to server
Yaml file: use-nnet2: True decoder: use-threaded-decoder: True model : models/tdnn_sp/final.mdl word-syms : models/tdnn_sp/words.txt fst : models/tdnn_sp/HCLG.fst mfcc-config : models/conf/mfcc_hires.conf ivector-extraction-config : models/conf/ivector_extractor.conf phone-syms: models/phones.txt nnet-mode: 3 max-active: 10000 beam: 15.0 lattice-beam: 6.0 acoustic-scale: 1 frame-subsamping-factor : 3 do-endpointing : true endpoint-silence-phones : "1:2:3:4:5:6:7:8:9:10" traceback-period-in-secs: 0.25 chunk-length-in-secs: 0.25 num-nbest: 10
Additional functionality that you can play with:
out-dir: tmp
use-vad: False silence-timeout: 10
post-processor: perl -npe 'BEGIN {use IO::Handle; STDOUT->autoflush(1);} s/(.*)/\1./;'
full-post-processor: ./sample_full_post_processor.py
logging: version : 1 disable_existing_loggers: False formatters: simpleFormater: format: '%(asctime)s - %(levelname)7s: %(name)10s: %(message)s' datefmt: '%Y-%m-%d %H:%M:%S' handlers: console: class: logging.StreamHandler formatter: simpleFormater level: DEBUG root: level: DEBUG handlers: [console]