Closed Quasimondo closed 7 years ago
I was rebalancing the Lambda weights after 100 epoch in demo_256p in a separate code. I calculated the weights by computing the average of l0,l1,l2,l3,l4,l5 in the 100th epoch. I will integrate the codes for this. Btw, I do not find balancing the weights affects the visual quality much.
For demo_512p.py and demo_1024p.py, I just fine-tune for 20 or 5 epochs. I just collect the statistics in the 1st epoch and put the weights there.
In the paper it says: "The hyperparameters {λl} are set automatically. They are initialized to the inverse of the number of elements in each layer. After 100 epochs, {λl} are rescaled to normalize the expected contribution of each term kΦl(I) − Φl(g(L; θ)) k1 to the loss." - I am trying to find that part in the demo_256p.py code, but I fail to spot it. From what I can see the weights for p0 to p5 are fixed values.
Can you point me to the part where this is applied?
Also I am wondering if the difference between the weights in demo_256p.py:
and the ones in demo_512p.py and demo 1024p.py is intentional?