Clarification between sc_model.py and sb_model.py

[DanielG1010]

Hello, thank you a lot for the work you’ve done on this project. I’ve been using your model and it has been working very well. I have a question that I hope you could help clarify.

I understand that the vgg directory contains code to utilize the pretrained VGG model. However, I noticed that the training process is conducted with sc_model.py. Could you please explain how sc_model.py differs from sb_model.py?

I believe sb_model.py is the model described in the paper. If this is the case, could you provide some insight into why sc_model.py is specifically used for the summer2winter dataset?

[cyclomon]

Hi, thank you for your comment. both of sc_model.py and sb_model.py contains our proposed UNSB model, but sc_model.py file is used when using pre-trained VGG model.

We implemented the two models in different way, as extracting features from generator & VGG model show slight difference. However, overall training processes are same.

[DanielG1010]

Thank you for the clarification. Given that both sc_model.py and sb_model.py contain the proposed UNSB model, I’m interested in understanding how to adjust certain hyperparameters when using sc_model.py.

In particular, if one uses sc_model.py, how can we tweak the hyperparameters such as lambda_NCE and lambda_SB in the UNSB loss calculation? I noticed that in sc_model.py, the given hyperparameters are lambda_spatial, lambda_perceptual, lambda_style, lambda_identity, and lambda_gradient.

I would greatly appreciate if you could provide some guidance in this matter.

[cyclomon]

Hi, lambda_spatial in sc_model corresponds to lambda_NCE.

For hyperparameter setting, we followed the previous work of "The Spatially-Correlative Loss for Various Image Translation Tasks" (CVPR 21).

Although there are many losses, we did not use lambda_perceptual,lambda_style,lambda_identity, and lambda_gradient, as their default setting is 0.

[DanielG1010]

Hello, thank you for your response, from the code, I believe the overall loss calculation is done in sb_model.py in here like this:

self.loss_G = self.loss_G_GAN + self.opt.lambda_SB*self.loss_SB + self.opt.lambda_NCE*loss_NCE_both

And in the sc_model.pyis done in here like this:

self.loss_G = self.loss_style + self.loss_per + self.loss_G_s + self.loss_G_s_idt_B + self.loss_idt_B + self.loss_G_GAN + self.loss_ENT

which is simplified considering these lambdas are zero:

 self.loss_G = self.loss_G_s + self.loss_G_GAN + self.loss_ENT

It looks like self.loss_G_s correspond to self.loss_NCE and self.loss_ENT correspond to self.loss_SB. As you mentioned the lambda_NCE appears as lambda_spatial, so I wonder if self.opt.lambda_SB and self.opt.lambda_GAN could be multiplied to the corresponding values (self.loss_ENT and self.loss_G_GAN) in sc_model.py

I might be wrong or missed something but could you indicate if that would be appropriate?