loss multiply 15

[lilipopololo]

I have tried to train this net when I set batchsize==8 and trained in 20 epoch,but I can't understand why the training loss don't decline and surrounding 10%. I can see the loss is in losses/unsup_loss and combine with a Vgg pretrain block ,the loss is 12 * self.reconstr_loss+6 * self.ssim_loss + 0.05 * self.smooth_loss (0.8*x+0.2*y+0.05*z)*15

I can't understand why the loss must multiply by 15.

I hope somebody can explain this,3U

[ToughStoneX]

Hello, I remember that the loss multiplied by 15 was commented which was not used in the training. It was originally used during debugging. The corresponding code in JDACS is: self.unsup_loss = 12 * self.reconstr_loss + 6 * self.ssim_loss + 0.18 * self.smooth_loss. The corresponding part in JDACS-MS is: self.unsup_loss = 12 * self.reconstr_loss + 6 * self.ssim_loss + 0.05 * self.smooth_loss. In general, the training loss should decline under the default setting. You may find out that the weight of the self.smooth_loss is modified according to different frameworks and backbones. The reason is that the smooth_loss term may hinder the convergence of self-supervised training if an inappropriate weight is used. In intuition, in the early stage of training, if the smooth_loss is too large, which provides a strong regularization to self-supervised training, the training loss may fall into a trivial solution rather can converge to a stable status. As a result, the trade-off between each term in self-supervision requires cautious alteration. Could you provide the curve of each component in the training loss, especially the smooth_loss?