search

dafei-qin / NFR_pytorch

Offical repo of "Neural Face Rigging for Animating and Retargeting Facial Meshes in the Wild"
57 stars 10 forks source link

loss function about z_i #11

Open chacorp opened 5 months ago

chacorp commented 5 months ago

Hi, thanks for the great work!

As I read through the paper I noticed there is no specific loss function dedicated to the z_i , the output from the identity encoder.

Considering that the size of the identity space of the ICT face model and the dimension of z_i are both 100, it seems that L2 loss can be used between the GT id from the ICT face model and the predicted z_i. However, in this case, it is a bit ambiguous because there is no corresponding GT for multiface data! I wonder if you actually did as I mentioned above or if you used a different method.

I'd appreciate for the reply :)

dafei-qin commented 3 months ago

Thanks for your interest in our work!

Yes we didn't apply explicit loss on the z_i, the identity encoder receives gradients only from the decoder loss.

The identity part is a relatively unexplored area in this work, where learning strategies like explicit loss or contrastive learning on the ICT dataset may be potential.

It's also an interesting direction to explore how this identity training on ICT affects that of Multiface. 😊