Question about ablation experiments

[congxin0920]

Hello, first of all, thank you very much for your work. I have a question that makes me very confused, and I hope to get your answer: I want to try one of the ablation experiments you wrote about in your paper, replacing the SeCu loss with the standard cross entropy loss, and I just change the calculation of loss_c as follows: loss_proj_c += criterion(proj_c1 / self.tw, label) + criterion(proj_c2 / self.tw, label), without changing any other Settings. The result is very strange and I think I misunderstood it. I would be very grateful if you could give me a little more detailed guidance

[qian-qi]

Hi,

Can you please elaborate your result? Is it for CIFAR or ImageNet?

[HaoyuWang0051]

Hello, I also encountered the same problem, aiming to use standard cross entropy loss, but the training results collapsed and eventually the acc diverged. I would like to ask what should be done with CE's training mode. The data set used is CIFAR.

[HaoyuWang0051]

And then this is the final training result。

[qian-qi]

Hi,

Training with standard cross-entropy loss is unstable and can collapse. This phenomenon is also observed on ImageNet. Therefore, we proposed SeCu to mitigate the problem.

[HaoyuWang0051]

Hello, thank you for your reply. In fact, I want to try to reproduce the CE results in the ablation experiment based on your open source code, as shown in the figure. But there has been no effect similar to the paper, so I would like to ask how to adjust the code structure for this problem.

[HaoyuWang0051]

Then here is the paper picture

[qian-qi]

Please try the modification as congxin0920.

[qian-qi]

I will close the issue and you can reopen it if there are any further questions.