UttaranB127
commented
3 years ago Hi Xingchen,
You might be looking at an earlier version of the paper, which had some typos. I'd recommend looking at the final published version: https://aaai.org/ojs/index.php/AAAI/article/view/5490. We actually did report the confusion matrix on the full dataset, which, in hindsight, is not the most accurate style of reporting. I would recommend reporting only on the test set, as we have done in subsequent works. However, the reported numbers in this paper are fair since all the methods were evaluated on the full dataset.
And yes, we had way more happy samples in the dataset compared to other classes. We balanced each batch during training by forcing that batch to contain samples from all the classes. Reporting only the accuracy indeed doesn't show the full picture, which is why we also show the confusion matrix. Another useful metric is the weighted accuracy: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.balanced_accuracy_score.html
xingchenzhang
vigneshrk29
Hi,
Thank you for your work and code!
I have a question on Fig. 6 of your paper.
In the caption, you said you were showing the results over the 3177 gaits. But in the figure, the total number of 4 classes (Angry, 1073, Happy 565, Sad 65 and Neutral 232) are 1935 not 3177. In adition, I think you trained the model on E-gaits dataset. In this case, are you reporting results on all the E-gaits not only on the test set? Because I think your test set does not have so many samples.
Finally, as shown in Fig.6, your sample classes are highly imbalanced. I am wondering if your training set is also imbalanced, and in this case, how much sense the accuracy can be used as the evaluation metric.
Sorry for so many questions. I am not sure if I misunderstood something. Could you please kindly help to clarify?
Many thanks! Xingchen