About al_loss

[xmengxin]

In trainers.py line 117-120 features, similarity, _ = self.net(imgs) features_target, similaritytarget, = self.net(imgs_target) scores = similarity * self.args.scala_ce loss_source = self.al_loss(scores, labels) The al_loss is about labels and scores. In this code, scores is made from imgs, but in the section3.4 of the paper, it is from image_target. The author said al_loss is minimized for the auxiliary dataset. Could you explain it? I got confused.

[KovenYu]

Hi @9meng9, thanks for your attention. Auxiliary dataset (in the paper) refers to source dataset in the code. In section 3.4 L_{AL} is computed using the source images which correspond to imgs in the code.

[ZhangYuef]

Thanks for your sharing. @KovenYu

I have a question here that what's the purpose of arg.scala_ce in al_loss. Found here link.

I didn't find the corresponding part of this in the paper.

[KovenYu]

Thanks for your attention. @ZhangYuef

Quoting from Sec. 4.2 from our paper:

Since optimizing entropy-based loss L_{AL} with the unit norm constraint has convergence issue, we follow the training method in [1], ..., we enforce the constraint to start our model learning and multiply the constrained inner products by the average inner product value in the pretraining.

Here, arg.scala_ce=30 is the average inner product value in the pretraining. This is to solve the softmax saturation problem [1].

[1] F. Wang, X. Xiang, J. Cheng, and A. L. Yuille. Normface: l2 hypersphere embedding for face verification. In ACMMM, 2017

[ZhangYuef]

Thanks for your attention. @ZhangYuef

Quoting from Sec. 4.2 from our paper:

Since optimizing entropy-based loss L_{AL} with the unit norm constraint has convergence issue, we follow the training method in [1], ..., we enforce the constraint to start our model learning and multiply the constrained inner products by the average inner product value in the pretraining.

Here, arg.scala_ce=30 is the average inner product value in the pretraining. This is to solve the softmax saturation problem [1].

[1] F. Wang, X. Xiang, J. Cheng, and A. L. Yuille. Normface: l2 hypersphere embedding for face verification. In ACMMM, 2017

Thanks! This solved my confusion : )