tdnn error

[sih4sing5hong5]

https://github.com/kaldi-asr/kaldi/blob/6016c004b117d03cf6368f0a8f506a554da63959/egs/wsj/s5/local/chain/tuning/run_tdnn_1f.sh https://github.com/kaldi-asr/kaldi/blob/6016c004b117d03cf6368f0a8f506a554da63959/egs/wsj/s5/local/nnet3/run_ivector_common.sh

https://github.com/kaldi-asr/kaldi/pull/2477/files

2018-06-27 04:44:20,134 [steps/nnet3/chain/train.py:563 - train - INFO ] Doing f
inal combination to produce final.mdl
2018-06-27 04:44:20,134 [steps/libs/nnet3/train/chain_objf/acoustic_model.py:570
 - combine_models - INFO ] Combining set([15, 16, 17, 18, 19, 20, 21, 22, 23, 24
, 25, 26, 27, 28]) models.
run.pl: job failed, log is in exp/chain/tdnn1f_sp/log/combine.log
Traceback (most recent call last):
  File "steps/nnet3/chain/train.py", line 622, in main
    train(args, run_opts)
  File "steps/nnet3/chain/train.py", line 574, in train
    use_multitask_egs=use_multitask_egs)
  File "steps/libs/nnet3/train/chain_objf/acoustic_model.py", line 621, in combi
ne_models
    scp_or_ark=scp_or_ark, egs_suffix=egs_suffix))
  File "steps/libs/common.py", line 152, in execute_command
    p.returncode, command))
Exception: Command exited with status 1: run.pl --mem 4G --gpu 1 exp/chain/tdnn1
f_sp/log/combine.log                 nnet3-chain-combine                 --max-o
bjective-evaluations=30    

exp/chain/tdnn1f_sp/log/combine.log

LOG (nnet3-chain-merge-egs[5.4.163~1402-04b1f]:PrintAggregateStats():nnet-exampl
e-utils.cc:1139) Processed 81 egs of avg. size 158.7 into 0 minibatches, discard
ing 100% of egs.  Avg minibatch size was -nan, #distinct types of egs/minibatche
s was 3/0
LOG (nnet3-chain-combine[5.4.163~1402-04b1f]:main():nnet3-chain-combine.cc:165) 
Read 0 examples.
ASSERTION_FAILED (nnet3-chain-combine[5.4.163~1402-04b1f]:main():nnet3-chain-com
bine.cc:166) : '!egs.empty()' 

[ Stack-Trace: ]
nnet3-chain-combine() [0x12a587e]
kaldi::MessageLogger::HandleMessage(kaldi::LogMessageEnvelope const&, char const
*)
kaldi::MessageLogger::~MessageLogger()
kaldi::KaldiAssertFailure_(char const*, char const*, int, char const*)
main
__libc_start_main
_start

ERROR (nnet3-am-copy[5.4.163~1402-04b1f]:ExpectToken():io-funcs.cc:203) Failed to read token [started at file position -1], expected <Nnet3>

[ Stack-Trace: ]
nnet3-am-copy() [0x9b939c]
kaldi::MessageLogger::HandleMessage(kaldi::LogMessageEnvelope const&, char const*)
kaldi::MessageLogger::~MessageLogger()
kaldi::ExpectToken(std::istream&, bool, char const*)
kaldi::nnet3::Nnet::Read(std::istream&, bool)
void kaldi::ReadKaldiObject<kaldi::nnet3::Nnet>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, kaldi::nnet3::Nnet*)
main
__libc_start_main
_start

[sih4sing5hong5]

egs/log/create_valid_subset.log

WARNING (nnet3-chain-get-egs[5.4.163~1402-04b1f]:ProcessFile():nnet3-chain-get-e
gs.cc:124) Not producing egs for utterance sp1.1-0000598TW02M1P0276-tong0116592-
ku0000000 because it is too short: 63 frames.
WARNING (nnet3-chain-get-egs[5.4.163~1402-04b1f]:main():nnet3-chain-get-egs.cc:425) 688 utterances had errors and could not be processed.

exp/chain/tdnn1f_sp/egs/log/get_egs.2.log

WARNING (nnet3-chain-get-egs[5.4.163~1402-04b1f]:ProcessFile():nnet3-chain-get-e
gs.cc:124) Not producing egs for utterance sp0.9-0000179TW02M0P0083-tong0058271-
ku0000000 because it is too short: 94 frames.
WARNING (nnet3-chain-get-egs[5.4.163~1402-04b1f]:ProcessFile():nnet3-chain-get-e
gs.cc:124) Not producing egs for utterance sp0.9-0000179TW02M0P0083-tong0058272-
ku0000000 because it is too short: 74 frames.
LOG (apply-cmvn[5.4.163~1402-04b1f]:main():apply-cmvn.cc:81) Copied 87301 uttera
nces.