[Question] Loss Factor to Weight LoRA Loss and VSD Loss

thu-ml / prolificdreamer

ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation (NeurIPS 2023 Spotlight)

https://ml.cs.tsinghua.edu.cn/prolificdreamer/

Apache License 2.0

1.47k stars 44 forks source link

[Question] Loss Factor to Weight LoRA Loss and VSD Loss #8

Open jameskuma opened 1 year ago

jameskuma commented 1 year ago

Thank you for your awesome research!!!

I have been trying to implement prolific dreamer recently. Would you like to share more details about how to set factors for VSD loss and LoRA loss here.

Best, Kuma

thuwzy commented 1 year ago

Thank you for your interest in our work! The factor of VSD loss and LoRA loss does not matter, since they optimize different parameters. To balance these two loss, setting the learning rates for them is important.