search

songrise / AvatarCraft

[ICCV23] AvatarCraft: Transforming Text into Neural Human Avatars with Parameterized Shape and Pose Control
Other
181 stars 7 forks source link

How do I understand the stylization loss in the Code in? #13

Closed HL4214 closed 1 year ago

HL4214 commented 1 year ago

In the paper, the total loss is the sum of sds loss,sil loss and eikonal loss. But I find a new loss named stylization loss in released code,I don't see what this loss does. Could you please explain the stylization loss?

songrise commented 1 year ago

Hi, sorry for the confusion. The “stylization loss” in the code means the SDS loss, they are the same. It is defined on line 349 of the stylize.py. To avoid OOM, we first calculate the gradient over whole rendered image using the SDS loss, and save the gradient for later back propagation. Help this answers your question.

songrise commented 1 year ago

FYI, SDS grad calculation is defined in https://github.com/songrise/AvatarCraft/blob/190a4bb07ac1959d50ad72563a908c8bc4eddc47/models/diffusion.py#L92C25-L92C25

HL4214 commented 1 year ago

I understand,Thank you very mach. I ignored optimizer.zero_grad(), which led me to think SDS loss was backward.