search

yuval-alaluf / SAM

Official Implementation for "Only a Matter of Style: Age Transformation Using a Style-Based Regression Model" (SIGGRAPH 2021) https://arxiv.org/abs/2102.02754
https://yuval-alaluf.github.io/SAM/
MIT License
632 stars 151 forks source link

How the loss lambdas are set? #54

Closed ytz123456 closed 1 year ago

ytz123456 commented 1 year ago

Hi @yuval-alaluf , thanks for the amazing work! As a research beginner, I am trying to implement a network structure similar to this one. The training of my model makes me struggle as I can't even get the training loss to go down. I have tried different hyper parameters and suspect that the problem is with my loss weights. May I know how did you search for the loss weights to train the model successfully?

yuval-alaluf commented 1 year ago

Defining the loss weights is mainly a heuristic search. What I would recommend doing is starting by trying to overfit on a small set of images (e..g, 10 images). This will help converge the training much faster. Once you're able to get good results on a few set of images, you can try training on more images and adjust the weights if needed.

ytz123456 commented 1 year ago

Thank you for your kind reply!

jkdev100 commented 8 months ago

Hi @yuval-alaluf, Excellent research and thanks for the above suggestion! What did you look for when adjusting loss weights up or down? And how did you finally decide that this set of loss weights was optimal / good?