search

microsoft / Deep3DFaceReconstruction

Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Set (CVPRW 2019)
MIT License
2.16k stars 441 forks source link

Confidence Net in the model #182

Open HOMGH opened 2 years ago

HOMGH commented 2 years ago

Hey, Thanks for your great work. Just wondering what is the notation of Confidence subnet (C-Net) in your repository? I can't find any C-Net in your codes. I'm pretty much sure that there were R-Net and C-Net in your codes (maybe the initial version)! But now it's just R_Net and Perceptual_Net in networks.py.

YuDeng commented 2 years ago

Hi, sorry that the current code does not include the C-Net part. Actually, we did not release the C-Net part in any previous versions.

HOMGH commented 2 years ago

Thanks for your reply. So, it means you do not use Confidence Net in your model? or you've used Confidence Net in your model but haven't released it's code?!

YuDeng commented 2 years ago

Well, in the paper we did use the C-Net and report the evaluation results. Later in practice, we find that in most of the times C-Net only has minor improvement (in quantitative metrics) upon the naive average shape and shows no difference in visual quality. Therefore we only release the single image part (R-Net) which we think is enough to produce good result.

HOMGH commented 2 years ago

Gotcha! Thanks for your clarification!

paolalf commented 2 years ago

@YuDeng , I have a question related to this topic. I was wondering what do you mean with "naive average shape". I am trying to reconstruct a face from multiple images of the same subject and I have managed to do so by getting the faceshape from single images of the subject (from different angles), and then I perform the average by using np.mean() between the "faceshape"s. This way I get a decent result, but the final face appears to be "squeezed", so I don't think that the np.mean() is what you mean with naive average shape. Thank you in advance.

YuDeng commented 2 years ago

Hi, the "average shape" means taking the average of identity coefficients and use the averaged coeff to construct the shape.

paolalf commented 2 years ago

Hi, the "average shape" means taking the average of identity coefficients and use the averaged coeff to construct the shape.

Thank you very much for your answer, @YuDeng . I have another question, If you could help me. Assuming that I have n pictures of the same subject, if I wanted to take the average of the identity coefficients, what I understand is therefore that I have to "collect" the n coeff_s returned by every

coeff_,face_shape_,face_texture_,face_color_,landmarks_2d_,recon_img_,tri_ = sess.run([coeff,\
                    face_shape,face_texture,face_color,landmarks_2d,recon_img,tri],feed_dict = {images: input_img})

performed on every image of my subject and take the average of the identity portion of said coeffs. This is then merged with the untouched portion of the coeff and finally what it's get is the averagedcoeff. Following this, do you have a suggestion on how to get the faceshape from the new averagedcoeff? Do I have to call again the above code, making sure that the tf.Graph() uses the new coefficient, maybe implementing an if/else statement within the tf.Graph()? Another thing that I tried was to execute:

FaceReconstructor.Reconstruction_Block(averaged_coeff,opt)
face_shape = FaceReconstructor.face_shape_t

within the tf.Session(), but it doesn't seem to be the right path to follow. Thank you again for your time.