Closed Phoebe-ovo closed 1 year ago
Thank you for your attention. In fact, we have v = 0 for the numerator, apparently. ;) We write this in the code for implementation convenience.
Can you elaborate on v=0 for the numerator? Since the final loss goss through cross-entropy function, it seems the numerator also needs to take the same value as its corresponding component in the denominator, which gives a lower bound.
Now I see it. The additional addition in the logic of the numerator is gone because it becomes zero upon evaluation due to equal weights for the same class.
In you paper, the logit in the molecular is not changed, only the logits in the denominator are changed. However, in the code, I find all the logits are changed. https://github.com/blackfeather-wang/ISDA-for-Deep-Networks/blob/b66a594482557dada126211d65a4e9b6f4328423/Image%20classification%20on%20ImageNet/ISDA_imagenet.py#L99