Class-wise contrastive learning loss

ChangyaoTian / VL-LTR

VL-LTR: Learning Class-wise Visual-Linguistic Representation for Long-Tailed Visual Recognition

61 stars 8 forks source link

Class-wise contrastive learning loss #13

Open Kevin-Planolles opened 5 months ago

Kevin-Planolles commented 5 months ago

The code is not clear about the CCL mentionned in the paper. It is not mentionned in the losses.py file. Is it possible to further explain where in the code this particular loss is used ?

ChangyaoTian commented 5 months ago

Sure. The CCL loss is implicitly implemented via the following functions:

The labels2idxs first generates the target matrix for the input samples, where samples from the same class will have positive target values;
Then the CCL loss is calculated in the PretrainSentLoss module here, where the base_criterion, (i.e. the LabelSmoothingCrossEntropy module) computes the loss of visual and linguistics branches respectively.