search

fgnt / pb_chime5

Speech enhancement system for the CHiME-5 dinner party scenario
MIT License
109 stars 34 forks source link

revert to use the eval data #14

Closed sw005320 closed 4 years ago

sw005320 commented 4 years ago

I think it's working now. It would be great if you review it and merge it.

boeddeker commented 4 years ago

Thank you. Looks good for me.

sw005320 commented 4 years ago

Thanks a lot!!!! Could you also test the track 1 GSS part in https://github.com/chimechallenge/kaldi_chime6/pull/14? We're almost there.

boeddeker commented 4 years ago

Sure. I will test it. At the moment I am testing if the performance for multiarray didn't change between my chime5 experiment and the implementation in kaldi for chime6.

boeddeker commented 4 years ago

I tested now the multi-array GSS in track 1 for the DEV dataset and got a WER of 62.85% with the baseline CHiME-5 ASR system (i.e. not with the new ASR). This is slightly better than the WER that I got with the CHiME-5 data. There the WER was 63.51%. So at least on the DEV data, the enhancement works.

Now I will also test the EVAL data.

sw005320 commented 4 years ago

Thanks. I'm relieved to hear that. I hope we'd not have some issues on the eval set, as well.

boeddeker commented 4 years ago

For the EVAL data the WER decreased. This is a bit strange, since the better synchronization should have no effect or slightly improve the WER. %WER 60.71 [ 33472 / 55132, 2551 ins, 14623 del, 16298 sub ] .../chime5/.../eval/wer_8_0.5 %WER 61.60 [ 33964 / 55132, 2608 ins, 15187 del, 16169 sub ] .../chime6/.../eval/wer_8_0.5