search

Jiaxin-Ye / TIM-Net_SER

[ICASSP 2023] Official Tensorflow implementation of "Temporal Modeling Matters: A Novel Temporal Emotional Modeling Approach for Speech Emotion Recognition".
GNU General Public License v3.0
161 stars 25 forks source link

Test set data seen during training #16

Closed adrianastan closed 1 year ago

adrianastan commented 1 year ago

Hi,

I just noticed that in your code you are presenting the test set data to the model.fit() method and select the best model based on it, instead of using a separate validation set or early stopping/fixed number of epochs.

However, this means that your reported results on the test set are "cherry picked". Do you have a report on the performance of your model when the test data is not used for validation during training? The same method seems to have been applied to your previous papers on SER.

Thanks, Adriana

Jiaxin-Ye commented 1 year ago

Dear Dr. Stan,

Thank you for your interest in this project and your valuable suggestions. I sincerely apologize for any inconvenience caused to you. Upon carefully rechecking the code, I have finished the experimental supplementary results and will update them on GitHub and arXiv in the next few days.

These are our supplementary results under 10-fold cross-validation (UAR % / WAR %), and all models are trained with the original hyperparameters for 500 epochs, which are evaluated only in the last epoch to avoid overfitting issues. The updated README file provides further details on the experiment specifics. Feel free to refer to it for more information.

<!DOCTYPE html>

UAR (%)/WAR (%) CASIA EMODB EMOVO IEMOCAP RAVDESS SAVEE
TIM-Net 91.08 / 91.08 89.19 / 90.28 86.56 / 86.56 69.00 / 68.29 90.04 / 90.07 77.26 / 79.36

I greatly appreciate your feedback, which helps us continually refine and enhance the quality of our work. If you have any further questions or feedback, please don't hesitate to reach out to me. Thank you very much for your understanding and support.

Best regards,

Jiaxin Ye