search

CQFIO / PhotographicImageSynthesis

Photographic Image Synthesis with Cascaded Refinement Networks
https://cqf.io/ImageSynthesis/
1.25k stars 226 forks source link

Some details about the loss weight #13

Closed MORIZHIJIANDX closed 6 years ago

MORIZHIJIANDX commented 6 years ago

I have some questions about the loss weight . 1.At the beginning of the training, the loss weights are "initialized to the inverse of the number of elements in each layer", "the number of elements in each layer" refers to the number of elements of the future map mi,which is extracted by the layer i ?(or the elements of weight of layer i ?) So the initial value of the loss weight wi is: wi = 1/count(mi) 2.Keep the loss weight constant and then train 100 times.In the last training we got the loss L(ik) of layer i and image k. So the final weight wi is: wi = ( Li1 + Li2 +... + Lin)/n I am not sure whether I understand it correctly.Look forward to your reply.

CQFIO commented 6 years ago

  1. In the code, I use reduced tf.reduce_mean and your understanding is right. wi = 1/count(mi)

  2. It is also basically correct. I rescale the terms according to ( Li1 + Li2 +... + Lin)/n.

In fact, you can just use the weight I put in the code and it should work out of the box.