search

Snowdar / asv-subtools

An Open Source Tools for Speaker Recognition
Apache License 2.0
590 stars 135 forks source link

执行subtools/scoreSets.sh出错 #17

Closed lanhaile closed 2 years ago

lanhaile commented 3 years ago

Hi， Snowdar! 我在ap-olr2020-baseline中，通过run_pytorch_xvector.py生成了xvector，现在我想用subtools/scoreSets.sh对我的enrollsets和testsets进行打分（我的数据集都是一个speaker只有一个utterance），但现在出现了如下错误提示： **[Auto find] Your vectortype is xvector

[Notice] It will set the default config task3_enroll[task3_enroll task3_enroll task3_test] for lda, submean and whiten, if used. allsets:task3_enroll task3_test task3_enroll task3_enroll task3_enroll task3_test task3_enroll task3_enroll task3_enroll task3_test
[ lr ] ivector-normalize-length --scaleup=false scp:exp/pytorch_xvector/far_epoch_21/task3_enroll/xvector.scp ark:exp/pytorch_xvector/far_epoch_21/task3_enroll/xvector_norm.ark LOG (ivector-normalize-length[5.5.804~1-a8c6]:main():ivector-normalize-length.cc:90) Processed 21580 iVectors. LOG (ivector-normalize-length[5.5.804~1-a8c6]:main():ivector-normalize-length.cc:94) Average ratio of iVector to expected length was 44.3168, standard deviation was 3.67065 ivector-compute-lda --dim=100 --total-covariance-factor=0.1 ark:exp/pytorch_xvector/far_epoch_21/task3_enroll/xvector_norm.ark ark:data/mfcc_20_5.0/task3_enroll/utt2spk exp/pytorch_xvector/far_epoch_21/task3_enroll/transform_100.mat LOG (ivector-compute-lda[5.5.804~1-a8c6]:main():ivector-compute-lda.cc:288) Read 21580 utterances, 0 with errors. LOG (ivector-compute-lda[5.5.804~1-a8c6]:main():ivector-compute-lda.cc:294) Computing within-class covariance. LOG (ivector-compute-lda[5.5.804~1-a8c6]:main():ivector-compute-lda.cc:299) 2-norm of iVector mean is 0.771824 LOG (ivector-compute-lda[5.5.804~1-a8c6]:ComputeLdaTransform():ivector-compute-lda.cc:136) Stats have 21580 speakers, 21580 utterances. ASSERTION_FAILED (ivector-compute-lda[5.5.804~1-a8c6]:ComputeLdaTransform():ivector-compute-lda.cc:137) Assertion failed: (!stats.Empty())

[ Stack-Trace: ] /home/lanhaile/kaldi/src/lib/libkaldi-base.so(kaldi::MessageLogger::LogMessage() const+0x808) [0x7f4b1168e35c] /home/lanhaile/kaldi/src/lib/libkaldi-base.so(kaldi::KaldiAssertFailure_(char const, char const, int, char const)+0x59) [0x7f4b1168eda3] ivector-compute-lda(kaldi::ComputeLdaTransform(std::map<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, kaldi::Vector, std::less<std::cxx11::basic_string<char, std::char_traits, std::allocator > >, std::allocator<std::pair<std::cxx11::basic_string<char, std::char_traits, std::allocator > const, kaldi::Vector> > > const&, std::map<std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::vector<std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::cxx11::basic_string<char, std::char_traits, std::allocator > > >, std::less<std::cxx11::basic_string<char, std::char_traits, std::allocator > >, std::allocator<std::pair<std::cxx11::basic_string<char, std::char_traits, std::allocator > const, std::vector<std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > > > > const&, float, float, kaldi::MatrixBase)+0x705) [0x40d38f] ivector-compute-lda(main+0xd6e) [0x40e6eb] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0) [0x7f4b09e2a840] ivector-compute-lda(_start+0x29) [0x40ca89]

ivector-transform exp/pytorch_xvector/far_epoch_21/task3_enroll/ scp:exp/pytorch_xvector/far_epoch_21/task3_enroll/xvector.scp ark:exp/pytorch_xvector/far_epoch_21/task3_enroll/xvector_lda100.ark ERROR (ivector-transform[5.5.804~1-a8c6]:Read():kaldi-matrix.cc:1617) Failed to read matrix from stream. : Expected "[", got EOF File position at start is -1, currently -1

[ Stack-Trace: ] /home/lanhaile/kaldi/src/lib/libkaldi-base.so(kaldi::MessageLogger::LogMessage() const+0x808) [0x7fbf6bdd735c] ivector-transform(kaldi::MessageLogger::LogAndThrow::operator=(kaldi::MessageLogger const&)+0x11) [0x40a709] /home/lanhaile/kaldi/src/lib/libkaldi-matrix.so(kaldi::Matrix::Read(std::istream&, bool, bool)+0x1a82) [0x7fbf6c020f76] /home/lanhaile/kaldi/src/lib/libkaldi-util.so(void kaldi::ReadKaldiObject<kaldi::Matrix >(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, kaldi::Matrix*)+0x239) [0x7fbf6c298bdc] ivector-transform(main+0xeb) [0x409681] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0) [0x7fbf64573840] ivector-transform(_start+0x29) [0x4094c9]

kaldi::KaldiFatalErrorawk: cmd. line:1: fatal: cannot open file `exp/pytorch_xvector/far_epoch_21/task3_test/lr_task3_enroll_task3_test_lda100_submean_norm.score' for reading (No such file or directory) Tansforming score to table done.** 期待您的答疑，万分感谢!

Snowdar commented 3 years ago

你好， 不建议使用这个配置复杂的打分脚本，请参考kaldi sre16 recipe写一下自己的后端流程。

祝好！

On Jan 26, 2021, at 6:33 PM, lanhaile notifications@github.com wrote:

 Hi， Snowdar! 我在ap-olr2020-baseline中，通过run_pytorch_xvector.py生成了xvector，现在我想用subtools/scoreSets.sh对我的enrollsets和testsets进行打分（我的数据集都是一个speaker只有一个utterance），但现在出现了如下错误提示： **[Auto find] Your vectortype is xvector

[Notice] It will set the default config task3_enroll[task3_enroll task3_enroll task3_test] for lda, submean and whiten, if used. allsets:task3_enroll task3_test task3_enroll task3_enroll task3_enroll task3_test task3_enroll task3_enroll task3_enroll task3_test [ lr ] ivector-normalize-length --scaleup=false scp:exp/pytorch_xvector/far_epoch_21/task3_enroll/xvector.scp ark:exp/pytorch_xvector/far_epoch_21/task3_enroll/xvector_norm.ark LOG (ivector-normalize-length[5.5.8041-a8c6]:main():ivector-normalize-length.cc:90) Processed 21580 iVectors. LOG (ivector-normalize-length[5.5.8041-a8c6]:main():ivector-normalize-length.cc:94) Average ratio of iVector to expected length was 44.3168, standard deviation was 3.67065 ivector-compute-lda --dim=100 --total-covariance-factor=0.1 ark:exp/pytorch_xvector/far_epoch_21/task3_enroll/xvector_norm.ark ark:data/mfcc_20_5.0/task3_enroll/utt2spk exp/pytorch_xvector/far_epoch_21/task3_enroll/transform_100.mat LOG (ivector-compute-lda[5.5.8041-a8c6]:main():ivector-compute-lda.cc:288) Read 21580 utterances, 0 with errors. LOG (ivector-compute-lda[5.5.8041-a8c6]:main():ivector-compute-lda.cc:294) Computing within-class covariance. LOG (ivector-compute-lda[5.5.8041-a8c6]:main():ivector-compute-lda.cc:299) 2-norm of iVector mean is 0.771824 LOG (ivector-compute-lda[5.5.8041-a8c6]:ComputeLdaTransform():ivector-compute-lda.cc:136) Stats have 21580 speakers, 21580 utterances. ASSERTION_FAILED (ivector-compute-lda[5.5.804~1-a8c6]:ComputeLdaTransform():ivector-compute-lda.cc:137) Assertion failed: (!stats.Empty())

[ Stack-Trace: ] /home/lanhaile/kaldi/src/lib/libkaldi-base.so(kaldi::MessageLogger::LogMessage() const+0x808) [0x7f4b1168e35c] /home/lanhaile/kaldi/src/lib/libkaldi-base.so(kaldi::KaldiAssertFailure_(char const, char const, int, char const)+0x59) [0x7f4b1168eda3] ivector-compute-lda(kaldi::ComputeLdaTransform(std::map<std::cxx11::basic_string<char, std::char_traits, std::allocator >, kaldi::Vector, std::less<std::cxx11::basic_string<char, std::char_traits, std::allocator > >, std::allocator<std::pair<std::cxx11::basic_string<char, std::char_traits, std::allocator > const, kaldi::Vector> > > const&, std::map<std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::vector<std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::cxx11::basic_string<char, std::char_traits, std::allocator > > >, std::less<std::cxx11::basic_string<char, std::char_traits, std::allocator > >, std::allocator<std::pair<std::cxx11::basic_string<char, std::char_traits, std::allocator > const, std::vector<std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::cxx11::basic_string<char, std::char_traits, std::allocator > > > > > > const&, float, float, kaldi::MatrixBase)+0x705) [0x40d38f] ivector-compute-lda(main+0xd6e) [0x40e6eb] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0) [0x7f4b09e2a840] ivector-compute-lda(_start+0x29) [0x40ca89]

ivector-transform exp/pytorch_xvector/far_epoch_21/task3_enroll/ scp:exp/pytorch_xvector/far_epoch_21/task3_enroll/xvector.scp ark:exp/pytorch_xvector/far_epoch_21/task3_enroll/xvector_lda100.ark ERROR (ivector-transform[5.5.804~1-a8c6]:Read():kaldi-matrix.cc:1617) Failed to read matrix from stream. : Expected "[", got EOF File position at start is -1, currently -1

[ Stack-Trace: ] /home/lanhaile/kaldi/src/lib/libkaldi-base.so(kaldi::MessageLogger::LogMessage() const+0x808) [0x7fbf6bdd735c] ivector-transform(kaldi::MessageLogger::LogAndThrow::operator=(kaldi::MessageLogger const&)+0x11) [0x40a709] /home/lanhaile/kaldi/src/lib/libkaldi-matrix.so(kaldi::Matrix::Read(std::istream&, bool, bool)+0x1a82) [0x7fbf6c020f76] /home/lanhaile/kaldi/src/lib/libkaldi-util.so(void kaldi::ReadKaldiObject(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, kaldi::Matrix*)+0x239) [0x7fbf6c298bdc] ivector-transform(main+0xeb) [0x409681] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0) [0x7fbf64573840] ivector-transform(_start+0x29) [0x4094c9]

kaldi::KaldiFatalErrorawk: cmd. line:1: fatal: cannot open file `exp/pytorch_xvector/far_epoch_21/task3_test/lr_task3_enroll_task3_test_lda100_submean_norm.score' for reading (No such file or directory) Tansforming score to table done.** 期待您的答疑，万分感谢!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

lanhaile commented 3 years ago

Hi， Snowdar,感谢您的回复！ 我还是想尝试一下用scoreSets进行打分，感觉还差一点点就出结果了。 我现在已经执行到logistic-regression-eval，不过没有输出打分结果，也没有错误提示。执行情况如下： LOG (ivector-transform[5.5.804~1-a8c6]:main():ivector-transform.cc:91) Processed 19422 iVectors. LOG (ivector-transform[5.5.804~1-a8c6]:main():ivector-transform.cc:99) Norm of mean was 2208.33 (should be close to zero), length divided by sqrt(dim) was 118.788 (should probably be close to one) ivector-subtract-global-mean exp/pytorch_xvector/far_epoch_21/task3_enroll_volume_aug/xvector_lda100.global.vec ark:exp/pytorch_xvector/far_epoch_21/task3_test_volume_aug/xvector_lda100.ark ark:exp/pytorch_xvector/far_epoch_21/task3_test_volume_aug/xvector_lda100_submean.ark LOG (ivector-subtract-global-mean[5.5.804~1-a8c6]:main():ivector-subtract-global-mean.cc:108) Wrote 19422 mean-subtracted iVectors ivector-normalize-length --scaleup=false ark:exp/pytorch_xvector/far_epoch_21/task3_test_volume_aug/xvector_lda100_submean.ark ark:exp/pytorch_xvector/far_epoch_21/task3_test_volume_aug/xvector_lda100_submean_norm.ark LOG (ivector-normalize-length[5.5.804~1-a8c6]:main():ivector-normalize-length.cc:90) Processed 19422 iVectors. LOG (ivector-normalize-length[5.5.804~1-a8c6]:main():ivector-normalize-length.cc:94) Average ratio of iVector to expected length was 1183.48, standard deviation was 239.216 logistic-regression-train --max-steps=20 --mix-up=0 ark:exp/pytorch_xvector/far_epoch_21/task3_enroll_volume_aug/xvector_lda100_submean_norm.ark ark:data/mfcc_20_5.0/task3_enroll_volume_aug/utt2label exp/pytorch_xvector/far_epoch_21/task3_enroll_volume_aug/lr/lr.raw.model LOG (logistic-regression-train[5.5.804~1-a8c6]:main():logistic-regression-train.cc:90) Retrieved 3000 vectors with 0 missing. There were 1000 class labels. LOG (logistic-regression-train[5.5.804~1-a8c6]:DoStep():logistic-regression.cc:221) Objective function is -6.90801 LOG (logistic-regression-train[5.5.804~1-a8c6]:DoStep():logistic-regression.cc:221) Objective function is -6.90683 LOG (logistic-regression-train[5.5.804~1-a8c6]:DoStep():logistic-regression.cc:221) Objective function is -6.90606 LOG (logistic-regression-train[5.5.804~1-a8c6]:DoStep():logistic-regression.cc:221) Objective function is -6.90403 LOG (logistic-regression-train[5.5.804~1-a8c6]:DoStep():logistic-regression.cc:221) Objective function is -6.90004 LOG (logistic-regression-train[5.5.804~1-a8c6]:DoStep():logistic-regression.cc:221) Objective function is -6.89239 LOG (logistic-regression-train[5.5.804~1-a8c6]:DoStep():logistic-regression.cc:221) Objective function is -6.87764 LOG (logistic-regression-train[5.5.804~1-a8c6]:DoStep():logistic-regression.cc:221) Objective function is -6.84993 LOG (logistic-regression-train[5.5.804~1-a8c6]:DoStep():logistic-regression.cc:221) Objective function is -6.71212 LOG (logistic-regression-train[5.5.804~1-a8c6]:DoStep():logistic-regression.cc:221) Objective function is -6.71211 LOG (logistic-regression-train[5.5.804~1-a8c6]:DoStep():logistic-regression.cc:221) Objective function is -6.71211 LOG (logistic-regression-train[5.5.804~1-a8c6]:DoStep():logistic-regression.cc:221) Objective function is -6.71211 LOG (logistic-regression-train[5.5.804~1-a8c6]:DoStep():logistic-regression.cc:221) Objective function is -6.71211 LOG (logistic-regression-train[5.5.804~1-a8c6]:DoStep():logistic-regression.cc:221) Objective function is -6.71211 LOG (logistic-regression-train[5.5.804~1-a8c6]:DoStep():logistic-regression.cc:221) Objective function is -6.71211 LOG (logistic-regression-train[5.5.804~1-a8c6]:DoStep():logistic-regression.cc:221) Objective function is -6.71211 LOG (logistic-regression-train[5.5.804~1-a8c6]:DoStep():logistic-regression.cc:221) Objective function is -6.71211 LOG (logistic-regression-train[5.5.804~1-a8c6]:DoStep():logistic-regression.cc:221) Objective function is -6.71211 LOG (logistic-regression-train[5.5.804~1-a8c6]:DoStep():logistic-regression.cc:221) Objective function is -6.71211 LOG (logistic-regression-train[5.5.804~1-a8c6]:DoStep():logistic-regression.cc:221) Objective function is -6.71211 LOG (logistic-regression-train[5.5.804~1-a8c6]:Train():logistic-regression.cc:58) Finished training parameters without mixture components. logistic-regression-copy exp/pytorch_xvector/far_epoch_21/task3_enroll_volume_aug/lr/lr.raw.model exp/pytorch_xvector/far_epoch_21/task3_enroll_volume_aug/lr/lr.scale.model LOG (logistic-regression-copy[5.5.804~1-a8c6]:main():logistic-regression-copy.cc:71) Wrote model to exp/pytorch_xvector/far_epoch_21/task3_enroll_volume_aug/lr/lr.scale.model logistic-regression-eval --apply-log=true --max-steps=20 --mix-up=0 exp/pytorch_xvector/far_epoch_21/task3_enroll_volume_aug/lr/lr.scale.model ark:exp/pytorch_xvector/far_epoch_21/task3_enroll_volume_aug/lr/trials.label ark:exp/pytorch_xvector/far_epoch_21/task3_test_volume_aug/xvector_lda100_submean_norm.ark 在logistic-regression-eval的四个参数中，前三个输入参数文件，都正常生成，可以在我的目录下找到，有什么可能的原因导致不能输出结果？ 我在shell脚本上找不到是怎样调用logistic-regression-eval的，是与您说的复杂配置文件有关吗？ 期待您的答复，谢谢！