search

ToughStoneX / Self-Supervised-MVS

Pytorch codes for "Self-supervised Multi-view Stereo via Effective Co-Segmentation and Data-Augmentation"
152 stars 15 forks source link

How to set the weights of training loss? #8

Open ghost opened 3 years ago

ghost commented 3 years ago

Thanks for your wonderful work! I have some question about the loss in the paper. I train the JDACS with MVSNet according the the provided code, and the performance of the obtined model is much lower than that given in the paper on DTU dataset, the Acc., Comp., and overall I obtained are 0.6295, 0.5531, and 0.5913. I found that the setting of weights in the code is different from the setting of weights given in the paper. When I use the setting of weights given in the paper, the model doesn't seem to work, the thres2mm_error on test set is about 0.99 at the 1st epoch during the training. But the thres2mm_error on test set is about 0.55 at the 1st epoch during the training when I use the default setting of weights in the code. In this case, the weights of the loss seem very important. How could I set the weight to achieve the similar performance as the results provided in the paper?

ToughStoneX commented 3 years ago

Hi, thanks for the questions. There are several concerns about the experiments: 1, Could you provide detailed information about your environment? For example, GPUs, Pytorch version, and etc. 2, Have you tried checkpoints with different iterations rather than the default one? The performance may fluctuate in self-supervised training. For example, you may find that the curve of test error may fluctuate under different iterations. 3, In general, the default weights should be OK as long as the dataset is correctly organized. The current weights are further modified according to another repo unsup_mvs in order to provide a direct comparison. 4, If the questions still remain unsolved, please contact me and send the log of your checkpoint files to my e-mail: hongbinxu1013@gmail.com. Maybe I can try to find out the problems.

ewrfcas commented 2 years ago

Hi, I am also curious about the loss weights in the codes. '12,6,0.05' seems different from the setting of '0.8,0.2,0.0067' in the paper.

ToughStoneX commented 2 years ago

1, It is noted that each of the 3 components in the loss respectively represents the photometric consistency loss, the ssim loss, and the smoothness loss. In practice, the following 2 losses are used as extra regularizations, and the main component is the photometric consistency loss. Hence, to balance the effect of each loss, the weights are modified according to the scale of the first one. And the following ones can work fine unless the weights are set too large compared with the first one. 2, You may find similar settings here. We borrow the code of unsup_mvs (the first self-supervised MVS method) for a direct comparison. The reason is that they replace the basic photometric consistency with their customized self-supervision loss. Their loss is the sum of photometric difference of several views, which is larger than the original photometric loss. You may refer to their paper for more details. Due to the variation in scale, the weights are modified following them. 3, The weight '0.8,0.2,0.0067' is a legacy version for original photometric loss. In general, the weight '12,6,0.05' is paired with the new self-supervision loss of unsup_mvs, which can ensure the stable convergence of self-supervision.

ewrfcas commented 2 years ago

Thanks for your reply!