nesl / asvspoof2019

Our submission to the ASVspoof 2019: Automatic Speaker Verification Spoofing and Countermeasures Challenge
92 stars 40 forks source link

Performance gap #3

Open DanielMengLiu opened 4 years ago

DanielMengLiu commented 4 years ago

Dear authors,

After performing the experiments on PA with your code, we get EER gaps between your reported results and our implementation. Is there any place we need to careful about when we get the code?

Your interspeech results (spect feature on PA)
3.85 | 0.0960 | 3.81 | 0.0994 Our implementation with your code (feature=spect, epoch=200, lr=0.00005, batch_size=32) 5.15 | 0.1319 | 5.64 | 0.1433

Thanks.

wangziqi000 commented 4 years ago

Hi,

Firstly, the performance can be influenced by hyperparameter tuning. For example, we once used a batch size of 64 to reduce the randomness of each parameter update. I noticed that your learning rate is very small, maybe set it larger might help. Also, when using small learning rates, a higher number of epochs might be useful.

Secondly, training is affected by random factors, e.g. weight initialization. Even with the same training settings, results of two training can be different due to the randomness starting point. So I would suggest you do it multiple times and select the best one.

Best, Ziqi