search

mravanelli / pytorch-kaldi

pytorch-kaldi is a project for developing state-of-the-art DNN/RNN hybrid speech recognition systems. The DNN part is managed by pytorch, while feature extraction, label computation, and decoding are performed with the kaldi toolkit.
2.37k stars 446 forks source link

Use two feature to train ASR #103

Closed huangjun0413 closed 5 years ago

huangjun0413 commented 5 years ago

I use two features to train ASR model but get this error:

ERROR: the files /data2/kaldi-master/egs/aishell/s5_enhance/data_dev_ivector/feats.scp in fea_lst contain a different set of sentences!

The key values of the two features are the same.

the config file is: `[cfg_proto] cfg_proto = proto/global.proto cfg_proto_chunk = proto/global_chunk.proto

[exp] cmd = run_nn_script = run_nn out_folder = exp/aisehll_MLP_basic_ivector seed = 1234 use_cuda = True multi_gpu = False save_gpumem = False n_epochs_tr = 24

[dataset1] data_name = aishell_tr fea = fea_name=fbankpitch fea_lst=/data2/kaldi-master/egs/aishell/s5_enhance/data/train/feats.scp fea_opts=apply-cmvn --utt2spk=ark:/data2/kaldi-master/egs/aishell/s5_enhance/data/train/utt2spk scp:/data2/kaldi-master/egs/aishell/s5_enhance/data/train/cmvn.scp ark:- ark:- | add-deltas --delta-order=2 ark:- ark:- | cw_left=5 cw_right=0

    fea_name=ivector
    fea_lst=/data2/kaldi-master/egs/aishell/s5_enhance/data_train_ivector/feats.scp
    fea_opts=apply-cmvn --utt2spk=ark:/data2/kaldi-master/egs/aishell/s5_enhance/data_train_ivector/utt2spk  scp:/data2/kaldi-master/egs/aishell/s5_enhance/data_train_ivector/cmvn.scp ark:- ark:- | add-deltas --delta-order=0 ark:- ark:- |
    cw_left=0
    cw_right=0

lab = lab_name=lab_cd lab_folder=/data2/kaldi-master/egs/aishell/s5_enhance/exp/mono_ali lab_opts=ali-to-pdf lab_count_file=auto lab_data_folder=/data2/kaldi-master/egs/aishell/s5_enhance/data/train/ lab_graph=/data2/kaldi-master/egs/aishell/s5_enhance/exp/mono/graph

n_chunks = 50

[dataset2] data_name = aishell_dev fea = fea_name=fbankpitch fea_lst=/data2/kaldi-master/egs/aishell/s5_enhance/data/dev/feats.scp fea_opts=apply-cmvn --utt2spk=ark:/data2/kaldi-master/egs/aishell/s5_enhance/data/dev/utt2spk scp:/data2/kaldi-master/egs/aishell/s5_enhance/data/dev/cmvn.scp ark:- ark:- | add-deltas --delta-order=2 ark:- ark:- | cw_left=5 cw_right=0

    fea_name=ivector
    fea_lst=/data2/kaldi-master/egs/aishell/s5_enhance/data_dev_ivector/feats.scp
    fea_opts=apply-cmvn --utt2spk=ark:/data2/kaldi-master/egs/aishell/s5_enhance/data_dev_ivector/utt2spk  scp:/data2/kaldi-master/egs/aishell/s5_enhance/data_dev_ivector/cmvn.scp ark:- ark:- | add-deltas --delta-order=0 ark:- ark:- |
    cw_left=0
    cw_right=0

lab = lab_name=lab_cd lab_folder=/data2/kaldi-master/egs/aishell/s5_enhance/exp/mono_ali_dev lab_opts=ali-to-pdf lab_count_file=auto lab_data_folder=/data2/kaldi-master/egs/aishell/s5_enhance/data/dev lab_graph=/data2/kaldi-master/egs/aishell/s5_enhance/exp/mono/graph

n_chunks = 10

[dataset3] data_name = aishell_test fea = fea_name=fbankpitch fea_lst=/data2/kaldi-master/egs/aishell/s5_enhance/data/test/feats.scp fea_opts=apply-cmvn --utt2spk=ark:/data2/kaldi-master/egs/aishell/s5_enhance/data/test/test/utt2spk scp:/data2/kaldi-master/egs/aishell/s5_enhance/data/test/cmvn.scp ark:- ark:- | add-deltas --delta-order=2 ark:- ark:- | cw_left=5 cw_right=0

    fea_name=ivector
    fea_lst=/data2/kaldi-master/egs/aishell/s5_enhance/data_dev_ivector/feats.scp
    fea_opts=apply-cmvn --utt2spk=ark:/data2/kaldi-master/egs/aishell/s5_enhance/data_test_ivector/utt2spk  scp:/data2/kaldi-master/egs/aishell/s5_enhance/data_test_ivector/cmvn.scp ark:- ark:- | add-deltas --delta-order=0 ark:- ark:- |
    cw_left=0
    cw_right=0

lab = lab_name=lab_cd lab_folder=/data2/kaldi-master/egs/aishell/s5_enhance/exp/mono_ali_test lab_opts=ali-to-pdf lab_count_file=auto lab_data_folder=/data2/kaldi-master/egs/aishell/s5_enhance/data/test lab_graph=/data2/kaldi-master/egs/aishell/s5_enhance/exp/mono/graph

n_chunks = 10

[data_use] train_with = aishell_tr valid_with = aishell_dev forward_with = aishell_test

[batches] batch_size_train = 128 max_seq_length_train = 1000 increase_seq_length_train = False start_seq_len_train = 100 multply_factor_seq_len_train = 2 batch_size_valid = 128 max_seq_length_valid = 1000

[architecture1]arch_name = MLP_layers1 arch_proto = proto/MLP.proto arch_library = neural_networks arch_class = MLP arch_pretrain_file = none arch_freeze = False arch_seq_model = Falsednn_lay = 1024,1024,1024,100,1024,N_out_lab_cd dnn_drop = 0.15,0.15,0.15,0.15,0.15,0.0 dnn_use_laynorm_inp = False dnn_use_batchnorm_inp = False dnn_use_batchnorm = True,True,True,True,True,False dnn_use_laynorm = False,False,False,False,False,False dnn_act = relu,relu,relu,linear,relu,softmax arch_lr = 0.08 arch_halving_factor = 0.5 arch_improvement_threshold = 0.001 arch_opt = sgd opt_momentum = 0.0 opt_weight_decay = 0.0 opt_dampening = 0.0 opt_nesterov = False

[model] model_proto = proto/model.proto model = conc1=concatenate(fbankpitch,ivector) out_dnn1=compute(MLP_layers1,conc1) loss_final=cost_nll(out_dnn1,lab_cd) err_final=cost_err(out_dnn1,lab_cd)

[forward] forward_out = out_dnn1 normalize_posteriors = True normalize_with_counts_from = lab_cd save_out_file = False require_decoding = True

[decoding] decoding_script_folder = kaldi_decoding_scripts/ decoding_script = decode_dnn.sh decoding_proto = proto/decoding.proto min_active = 200 max_active = 7000 max_mem = 50000000 beam = 13.0 latbeam = 8.0 acwt = 0.2 max_arcs = -1 skip_scoring = false scoring_script = local/score.sh scoring_opts = "--min-lmwt 1 --max-lmwt 10" norm_vars = False

`

mravanelli commented 5 years ago

Hi, this error is raised in line 1131 of the file utils.py. Probably with some prints you could more easily figure out what is happening. It could be something trivial such as new lines at the end of the file or something like this.

On Tue, 28 May 2019 at 04:20, huang jun notifications@github.com wrote:

I use two features to train ASR model but get this error:

ERROR: the files /data2/kaldi-master/egs/aishell/s5_enhance/data_dev_ivector/feats.scp in fea_lst contain a different set of sentences!

The key values of the two features are the same.

the config file is: `[cfg_proto] cfg_proto = proto/global.proto cfg_proto_chunk = proto/global_chunk.proto

[exp] cmd = run_nn_script = run_nn out_folder = exp/aisehll_MLP_basic_ivector seed = 1234 use_cuda = True multi_gpu = False save_gpumem = False n_epochs_tr = 24

[dataset1] data_name = aishell_tr fea = fea_name=fbankpitch fea_lst=/data2/kaldi-master/egs/aishell/s5_enhance/data/train/feats.scp fea_opts=apply-cmvn --utt2spk=ark:/data2/kaldi-master/egs/aishell/s5_enhance/data/train/utt2spk scp:/data2/kaldi-master/egs/aishell/s5_enhance/data/train/cmvn.scp ark:- ark:- | add-deltas --delta-order=2 ark:- ark:- | cw_left=5 cw_right=0

fea_name=ivector
fea_lst=/data2/kaldi-master/egs/aishell/s5_enhance/data_train_ivector/feats.scp
fea_opts=apply-cmvn --utt2spk=ark:/data2/kaldi-master/egs/aishell/s5_enhance/data_train_ivector/utt2spk  scp:/data2/kaldi-master/egs/aishell/s5_enhance/data_train_ivector/cmvn.scp ark:- ark:- | add-deltas --delta-order=0 ark:- ark:- |
cw_left=0
cw_right=0

lab = lab_name=lab_cd lab_folder=/data2/kaldi-master/egs/aishell/s5_enhance/exp/mono_ali lab_opts=ali-to-pdf lab_count_file=auto lab_data_folder=/data2/kaldi-master/egs/aishell/s5_enhance/data/train/ lab_graph=/data2/kaldi-master/egs/aishell/s5_enhance/exp/mono/graph

n_chunks = 50

[dataset2] data_name = aishell_dev fea = fea_name=fbankpitch fea_lst=/data2/kaldi-master/egs/aishell/s5_enhance/data/dev/feats.scp fea_opts=apply-cmvn --utt2spk=ark:/data2/kaldi-master/egs/aishell/s5_enhance/data/dev/utt2spk scp:/data2/kaldi-master/egs/aishell/s5_enhance/data/dev/cmvn.scp ark:- ark:- | add-deltas --delta-order=2 ark:- ark:- | cw_left=5 cw_right=0

fea_name=ivector
fea_lst=/data2/kaldi-master/egs/aishell/s5_enhance/data_dev_ivector/feats.scp
fea_opts=apply-cmvn --utt2spk=ark:/data2/kaldi-master/egs/aishell/s5_enhance/data_dev_ivector/utt2spk  scp:/data2/kaldi-master/egs/aishell/s5_enhance/data_dev_ivector/cmvn.scp ark:- ark:- | add-deltas --delta-order=0 ark:- ark:- |
cw_left=0
cw_right=0

lab = lab_name=lab_cd lab_folder=/data2/kaldi-master/egs/aishell/s5_enhance/exp/mono_ali_dev lab_opts=ali-to-pdf lab_count_file=auto lab_data_folder=/data2/kaldi-master/egs/aishell/s5_enhance/data/dev lab_graph=/data2/kaldi-master/egs/aishell/s5_enhance/exp/mono/graph

n_chunks = 10

[dataset3] data_name = aishell_test fea = fea_name=fbankpitch fea_lst=/data2/kaldi-master/egs/aishell/s5_enhance/data/test/feats.scp fea_opts=apply-cmvn --utt2spk=ark:/data2/kaldi-master/egs/aishell/s5_enhance/data/test/test/utt2spk scp:/data2/kaldi-master/egs/aishell/s5_enhance/data/test/cmvn.scp ark:- ark:- | add-deltas --delta-order=2 ark:- ark:- | cw_left=5 cw_right=0

fea_name=ivector
fea_lst=/data2/kaldi-master/egs/aishell/s5_enhance/data_dev_ivector/feats.scp
fea_opts=apply-cmvn --utt2spk=ark:/data2/kaldi-master/egs/aishell/s5_enhance/data_test_ivector/utt2spk  scp:/data2/kaldi-master/egs/aishell/s5_enhance/data_test_ivector/cmvn.scp ark:- ark:- | add-deltas --delta-order=0 ark:- ark:- |
cw_left=0
cw_right=0

lab = lab_name=lab_cd lab_folder=/data2/kaldi-master/egs/aishell/s5_enhance/exp/mono_ali_test lab_opts=ali-to-pdf lab_count_file=auto lab_data_folder=/data2/kaldi-master/egs/aishell/s5_enhance/data/test lab_graph=/data2/kaldi-master/egs/aishell/s5_enhance/exp/mono/graph

n_chunks = 10

[data_use] train_with = aishell_tr valid_with = aishell_dev forward_with = aishell_test

[batches] batch_size_train = 128 max_seq_length_train = 1000 increase_seq_length_train = False start_seq_len_train = 100 multply_factor_seq_len_train = 2 batch_size_valid = 128 max_seq_length_valid = 1000

[architecture1]arch_name = MLP_layers1 arch_proto = proto/MLP.proto arch_library = neural_networks arch_class = MLP arch_pretrain_file = none arch_freeze = False arch_seq_model = Falsednn_lay = 1024,1024,1024,100,1024,N_out_lab_cd dnn_drop = 0.15,0.15,0.15,0.15,0.15,0.0 dnn_use_laynorm_inp = False dnn_use_batchnorm_inp = False dnn_use_batchnorm = True,True,True,True,True,False dnn_use_laynorm = False,False,False,False,False,False dnn_act = relu,relu,relu,linear,relu,softmax arch_lr = 0.08 arch_halving_factor = 0.5 arch_improvement_threshold = 0.001 arch_opt = sgd opt_momentum = 0.0 opt_weight_decay = 0.0 opt_dampening = 0.0 opt_nesterov = False

[model] model_proto = proto/model.proto model = conc1=concatenate(fbankpitch,ivector) out_dnn1=compute(MLP_layers1,conc1) loss_final=cost_nll(out_dnn1,lab_cd) err_final=cost_err(out_dnn1,lab_cd)

[forward] forward_out = out_dnn1 normalize_posteriors = True normalize_with_counts_from = lab_cd save_out_file = False require_decoding = True

[decoding] decoding_script_folder = kaldi_decoding_scripts/ decoding_script = decode_dnn.sh decoding_proto = proto/decoding.proto min_active = 200 max_active = 7000 max_mem = 50000000 beam = 13.0 latbeam = 8.0 acwt = 0.2 max_arcs = -1 skip_scoring = false scoring_script = local/score.sh scoring_opts = "--min-lmwt 1 --max-lmwt 10" norm_vars = False

`

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/mravanelli/pytorch-kaldi/issues/103?email_source=notifications&email_token=AEA2ZVSROIZEX4GLKMG5DTLPXTTNLA5CNFSM4HQA4ERKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4GWFEWEQ, or mute the thread https://github.com/notifications/unsubscribe-auth/AEA2ZVRHW7UB2QORIQHWEZ3PXTTNLANCNFSM4HQA4ERA .

huangjun0413 commented 5 years ago

yes, it works. Thank you very much.