search

blackfeather-wang / ISDA-for-Deep-Networks

An efficient implicit semantic augmentation method, complementary to existing non-semantic techniques.
582 stars 93 forks source link

Question about loss computation #24

Closed Phoebe-ovo closed 1 year ago

Phoebe-ovo commented 3 years ago

In you paper, the logit in the molecular is not changed, only the logits in the denominator are changed. However, in the code, I find all the logits are changed. https://github.com/blackfeather-wang/ISDA-for-Deep-Networks/blob/b66a594482557dada126211d65a4e9b6f4328423/Image%20classification%20on%20ImageNet/ISDA_imagenet.py#L99

blackfeather-wang commented 3 years ago

Thank you for your attention. In fact, we have v = 0 for the numerator, apparently. ;) We write this in the code for implementation convenience.

jackliu333 commented 2 years ago

Can you elaborate on v=0 for the numerator? Since the final loss goss through cross-entropy function, it seems the numerator also needs to take the same value as its corresponding component in the denominator, which gives a lower bound.

jackliu333 commented 2 years ago

Now I see it. The additional addition in the logic of the numerator is gone because it becomes zero upon evaluation due to equal weights for the same class.