Problem of pretrain

[ZJUHJ]

Thanks for you great work. I used the code and parameters provided by the author to pretrain, but I found that during the training process, loss_b collapses to a very small value very quickly and the zero-shot results on the val split of qvhighlight are also very poor. What is the reason for this? 2024_03_25_02_53_16 [Epoch] 001 [Loss] loss_b 0.0051 loss_g 0.4086 loss_f 0.1647 loss_s_inter 1.1177 loss_s_intra 1.1780 loss_overall 2.8742 2024_03_25_03_50_17 [Epoch] 002 [Loss] loss_b 0.0005 loss_g 0.3896 loss_f 0.1610 loss_s_inter 1.0108 loss_s_intra 1.1777 loss_overall 2.7396 2024_03_25_04_47_42 [Epoch] 003 [Loss] loss_b 0.0004 loss_g 0.3850 loss_f 0.1593 loss_s_inter 0.9755 loss_s_intra 1.1755 loss_overall 2.6957 2024_03_25_05_44_57 [Epoch] 004 [Loss] loss_b 0.0004 loss_g 0.3820 loss_f 0.1583 loss_s_inter 0.9541 loss_s_intra 1.1737 loss_overall 2.6685 2024_03_25_06_42_11 [Epoch] 005 [Loss] loss_b 0.0004 loss_g 0.3802 loss_f 0.1577 loss_s_inter 0.9385 loss_s_intra 1.1724 loss_overall 2.6491 2024_03_25_07_39_30 [Epoch] 006 [Loss] loss_b 0.0003 loss_g 0.3787 loss_f 0.1573 loss_s_inter 0.9261 loss_s_intra 1.1711 loss_overall 2.6336 2024_03_25_08_36_49 [Epoch] 007 [Loss] loss_b 0.0003 loss_g 0.3774 loss_f 0.1570 loss_s_inter 0.9158 loss_s_intra 1.1702 loss_overall 2.6208 2024_03_25_09_34_08 [Epoch] 008 [Loss] loss_b 0.0003 loss_g 0.3763 loss_f 0.1568 loss_s_inter 0.9068 loss_s_intra 1.1693 loss_overall 2.6094 2024_03_25_10_31_21 [Epoch] 009 [Loss] loss_b 0.0003 loss_g 0.3750 loss_f 0.1566 loss_s_inter 0.8992 loss_s_intra 1.1686 loss_overall 2.5997 2024_03_25_11_28_37 [Epoch] 010 [Loss] loss_b 0.0003 loss_g 0.3742 loss_f 0.1564 loss_s_inter 0.8920 loss_s_intra 1.1678 loss_overall 2.5907