Losses question

[BIGWangYuDong]

Hi! I felt confused about the weights for each loss, in your paper: loss = 5Lgan + Lssim + Lgrad + Lcyc however in the code, your loss is different: loss = Lgan + 5Lcyc + 0.25Lssim(each) + Lgrad(only one for fakeA)

I'm coding on pytorch, and don't know how to set the weights for each loss. https://github.com/honeygupta/UW-Net/blob/b95324a6691d0a152873a7df85ec1e2f99110aca/main.py#L203

Best regards, Dong

[honeygupta]

Hi Dong,

We tried different weights for the loss function and I guess the weights in the repo were something that we tried last. However, the results and the model provided were obtained with the weights mentioned in the paper.

Regards Honey

[BIGWangYuDong]

Dear Honey, Thanks for your answer, I had another question:

Have you tried using SSIM-Loss with SSIM(Rec_a, a) and SSIM(Rec_b, b) replace the 4 SSIM-Losses mentioned in your paper(SSIM(fake_a, b), SSIM(rec_b, fake_a), SSIM(fake_b, a), SSIM(rec_a, fake_b))?

Thanks, Dong

[honeygupta]

The main purpose of using SSIM is to make sure that the structure of the objects is preserved after style transfer. I don't recall using SSIM(Rec_a, a) as we thought that L1 should be enough. Since there are many loss terms, adding SSIM for the cyclic image seemed redundant.

Moreover, if you remove SSIM(fake_a, b), etc, then the style-transferred images, as well as the depth-map, become somewhat vague as the domains of the two datasets are very different and the generators tend to distort the objects to satisfy the adversarial loss. The adversarial loss is a feature-based loss and its minima depends on the discriminator's capacity. One thing we concluded from our experiments is that we need SSIM for the intermediate reconstructions for sure. You can definitely try adding SSIM(Rec_a, a) and SSIM(Rec_b, b), but I wouldn't suggest replacing.

Regards Honey