alumae / kaldi-gstreamer-server

Real-time full-duplex speech recognition server, based on the Kaldi toolkit and the GStreamer framwork.
BSD 2-Clause "Simplified" License
1.07k stars 342 forks source link

Worker fails loading Tedlium s5_r2 chain model ("Unknown component type TdnnComponent") #170

Closed dialogics closed 5 years ago

dialogics commented 5 years ago

I have trained the Tedlium s5_r2 model from Kaldi. Decoding on the test set works without problems, as does decoding audio using the model and the online2-wav-nnet3-latgen-faster binary from Kaldi.

When trying to load the model in the server I get the following error:

... 2019-01-10 13:25:59 - INFO: decoder2: Setting decoder property: frame-subsampling-factor = 3 2019-01-10 13:25:59 - INFO: decoder2: Setting decoder property: phone-syms = models/tedlium_en_extended/decoding/phones.txt 2019-01-10 13:25:59 - INFO: decoder2: Setting decoder property: max-active = 7000 2019-01-10 13:25:59 - INFO: decoder2: Setting decoder property: chunk-length-in-secs = 0.25 2019-01-10 13:25:59 - INFO: decoder2: Setting decoder property: fst = models/tedlium_en_extended/decoding/HCLG.fst 2019-01-10 13:26:00 - INFO: decoder2: Setting decoder property: model = models/tedlium_en_extended/decoding/final.mdl ERROR ([5.4.176~1-be967]:ReadNew():nnet-component-itf.cc:86) Unknown component type TdnnComponent

[ Stack-Trace: ] kaldi::MessageLogger::HandleMessage(kaldi::LogMessageEnvelope const&, char const*) kaldi::MessageLogger::~MessageLogger() kaldi::nnet3::Component::ReadNew(std::istream&, bool) kaldi::nnet3::Nnet::Read(std::istream&, bool) kaldi::nnet3::AmNnetSimple::Read(std::istream&, bool)

g_object_set_property

PyEval_EvalFrameEx . . . python() [0x4b988b] PyEval_EvalFrameEx PyEval_EvalFrameEx PyEval_EvalCodeEx python() [0x50160f] PyRun_FileExFlags PyRun_SimpleFileExFlags Py_Main __libc_start_main python() [0x497b8b] ...

The worker then fails to decode audio. I am using the Docker image of kaldi-gstreamer-server. I have used it with a self trained chain model from https://github.com/uhh-lt/kaldi-tuda-de/ without problems.

The YAML file used for starting:

use-nnet2: True decoder: nnet-mode: 3 use-threaded-decoder: True model : models/tedlium_en_extended/decoding/final.mdl word-syms : models/tedlium_en_extended/decoding/words.txt fst : models/tedlium_en_extended/decoding/HCLG.fst

feature-type : "mfcc"

mfcc-config : models/tedlium_en_extended/decoding/conf/mfcc.conf
frame-subsampling-factor: 3
ivector-extraction-config : models/tedlium_en_extended/decoding/conf/ivector_extractor.conf
#    max-active: 10000
#    beam: 10.0
max-active: 7000
beam: 15.0
lattice-beam: 6.0
acoustic-scale: 1.0 #0.083
do-endpointing : true
endpoint-silence-phones : "1:2:3:4:5:6:7:8:9:10"
traceback-period-in-secs: 0.25
chunk-length-in-secs: 0.25
num-nbest: 10
#Additional functionality that you can play with:
#lm-fst:  test/models/english/librispeech_nnet_a_online/G.fst
#big-lm-const-arpa: test/models/english/librispeech_nnet_a_online/G.carpa
phone-syms: models/tedlium_en_extended/decoding/phones.txt
#word-boundary-file: models/newModel/de_350k_nnet3chain_tdnn1f_1024_sp_bi/phones/word_boundary.int
#do-phone-alignment: true

out-dir: tmp

use-vad: False silence-timeout: 2

full-post-processor: kaldi-gstreamer-server/post_processor.py

logging: version : 1 disable_existing_loggers: False formatters: simpleFormater: format: '%(asctime)s - %(levelname)7s: %(name)10s: %(message)s' datefmt: '%Y-%m-%d %H:%M:%S' handlers: console: class: logging.StreamHandler formatter: simpleFormater level: DEBUG root: level: DEBUG handlers: [console]

Any help is much appreciated!

alumae commented 5 years ago

I suspect that the Kaldi DNN gstreamer plugin is compiled with and earlier version of Kaldi.

dialogics commented 5 years ago

The respective code has been added to Kaldi on 07.07.2017 so the prepared docker image is out of date for me. I am trying to build an up-to-date one right now.

Edit: Building a container with the current source did fix the problem.

KawhiZhao commented 4 years ago

The respective code has been added to Kaldi on 07.07.2017 so the prepared docker image is out of date for me. I am trying to build an up-to-date one right now.

Edit: Building a container with the current source did fix the problem.

Can you explain it in detail? I'm facing the same issue. Many thanks!