search

TideDancer / interspeech21_emotion

99 stars 20 forks source link

Results cannot be reproduced #7

Closed summersunshine1 closed 2 years ago

summersunshine1 commented 2 years ago

I tested 01F following the published code and have found that the wer and acc is low, which is 0.5 and 0.4, respectively. I don't know if I miss some important steps. May I discuss with you in detail?

TideDancer commented 2 years ago

Sure, I am happy to help. The 01F and 03F are indeed worst, but still should be > 0.73. Let me know the configs and learning curve of your run? For example, what is your final train-loss value? What is your alpha setting? How many epochs did you run?

summersunshine1 commented 2 years ago

Could we have a detailed discussion. Could you tell me the email. I will sent you my wechat number.

TideDancer commented 2 years ago

Sorry for the late reply. Sure, I can be reached at xingyu.develop.cai@gmail.com

taras-sereda commented 2 years ago

Hi @TideDancer. Interesting work! Do you have any plans to share a check point of a trained model? That way it will be much easier to verify correctness of results presented in a paper.

TideDancer commented 2 years ago

Hello @taras-sereda,

Thank you for your interests!

Sure, I have uploaded the checkpoints at https://drive.google.com/drive/folders/1Ndybde47HDy8O7aiNvT14pT7FqkqOsSX?usp=sharing. I also added a section "Reproduce using checkpoints" in the README, so you can follow the instructions there to download the checkpoints and run the evaluatiton script. You should get a final accuracy (combining all folds) slightly above 0.78.

Let me know if you encounter any problems reproducing or using the repo. Thanks again.