Jiaxin-Ye / TIM-Net_SER

[ICASSP 2023] Official Tensorflow implementation of "Temporal Modeling Matters: A Novel Temporal Emotional Modeling Approach for Speech Emotion Recognition".
GNU General Public License v3.0
161 stars 25 forks source link

Bug report about label smoothing (Model.py) #14

Closed NAM-hj closed 1 year ago

NAM-hj commented 1 year ago

For each iteration of k-th fold the y[train] should be smoothed once, but the y[train] were smoothed repeatedly over k-th iteration. The copy.deepcopy method can prevent this problem.

NAM-hj commented 1 year ago

1. Problem

Because of a list is a mutable object in Python, the following line can contaminate the target emotion one-hot array(y[train]). y[train]=smooth_labels(y[train], 0.1)

2. Log

I print the sum of y[train] for each iteration before smooth_labels function using the following line. print(f"i={i}, np.sum(y[train], axis=0)={np.sum(y[train], axis=0)}");

*Before deepcopy y[train] were smoothed again for each k-th iteration.

i=1, np.sum(y[train], axis=0)=[113.  73.  42.  65.  65.  73.  50.]
i=2, np.sum(y[train], axis=0)=[115.498856  68.999146  40.70008   60.50011   65.19978   73.2999 56.800034]
i=3, np.sum(y[train], axis=0)=[105.897545  72.597496  45.956913  61.436974  65.397125  70.16719 59.547146]
...

*After deepcopy Before label smoothing, the sum value of y[train] along the utterance axis should be an integer. Like this,

i=1, np.sum(y[train], axis=0)=[113.  73.  42.  65.  65.  73.  50.]
i=2, np.sum(y[train], axis=0)=[120.  69.  38.  60.  65.  74.  55.]
i=3, np.sum(y[train], axis=0)=[114.  73.  41.  60.  65.  71.  57.]
...
Jiaxin-Ye commented 1 year ago

Thank you for your valuable suggestion, we will update it in the next version. Thanks!!