Zhangjinso / PISE

123 stars 28 forks source link

The correspondence loss is actually not used #27

Closed imbinwang closed 3 years ago

imbinwang commented 3 years ago

Hi,

The correspondence loss of image generator in paper section 3.3 is not used in your code.

self.loss_names = [ 'app_gen','content_gen', 'style_gen', #'reg_gen', 'ad_gen', 'dis_img_gen', 'par', 'par1']

Does the correspondence loss not impact the final generated image? Thanks for your reply.

Zhangjinso commented 3 years ago

This is because we first trained the model with correspondence loss with “False" flag for spatio-aware module. Then we trained the spatio-aware module without this loss.

imbinwang commented 3 years ago

Thank for your clarification. As I understand, you means the training process consist of two steps:

  1. train the model with correspondence loss but without spatio-aware module,
  2. finetune the model saved in step1 with spatio-aware module but without correspondence loss.

Do you train the whole model in a process? It seems a valid choice.

Zhangjinso commented 3 years ago

The correspondence loss is used to bridge the gap between the source image and the target image in different poses. This is for computing the correspondence of these two features. You can check cocosnet. Though these two features do not have large gaps, but we use it to obtain a more precise correspondences. In fact, if do not consider the training time, you can train using spatio-aware module with the correspondence loss.

imbinwang commented 3 years ago

I will try it. Thank you.