search

tranluan / Nonlinear_Face_3DMM

Source code for "Nonlinear 3D Face Morphable Model"
http://cvlab.cse.msu.edu/project-nonlinear-3dmm.html
Apache License 2.0
676 stars 123 forks source link

Rendering Layer Not Differentiable. Cannot train with wild images. #59

Closed chaoshiedwin closed 4 years ago

chaoshiedwin commented 4 years ago

Hi @tranluan Thank you for your work. I am trying to fine-tune my model with wild images. But in your code, the z-buffer kernel is marked as NotDifferentiable, which means the reconstruction loss cannot be back propagated to the network. Hence wild images cannot be used for network training. This is inconsistent with what is claimed in the paper. Can you provide the gradient definition?

chaoshiedwin commented 4 years ago

Sorry, I didn't understand it correctly. The z-buffer does not need to be differentiable, z-buffer output is used to calculate the sampling position from texture map.

HOMGH commented 4 years ago

Sorry, I didn't understand it correctly. The z-buffer does not need to be differentiable, z-buffer output is used to calculate the sampling position from texture map.

@chaoshiedwin Hi, Could you train the model successfully? I trained the model on the 1/4 of training set and it seems it doesn't converge!

huyanfei-cqupt commented 3 years ago

Hi @tranluan Thank you for your work. I am trying to fine-tune my model with wild images. But in your code, the z-buffer kernel is marked as NotDifferentiable, which means the reconstruction loss cannot be back propagated to the network. Hence wild images cannot be used for network training. This is inconsistent with what is claimed in the paper. Can you provide the gradient definition?

hi,@chaoshiedwin Could you train the model successfully?