microsoft / Deep3DFaceReconstruction

Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Set (CVPRW 2019)
MIT License
2.21k stars 446 forks source link

What is the output of pre-trained model? #80

Closed nwatab closed 4 years ago

nwatab commented 4 years ago

Thank you, Yu for uploading pre-trained model. Uploaded model gets input image shaped in [1, 224, 224, 3] and outputs coefficient of [1, 258] for something. What is this 258 dimension coefficients for? I can see fc-id, fc-ex, fc-tex and some other coefficients of 3DMM model on tensorboard (Figure below). I expected those 3DMM coefficients lead to reconstructed image and losses, not another coefficients for something. Thank you. I welcome any comments.

Screenshot from 2020-06-16 17-02-25

YuDeng commented 4 years ago

In my later experiments, this dimension is used to predict the focal length of an image. It is useless here. It should be a mistake that I include this dimension in the final output. The first 257 dimensions are identity(80), expression(64), texture(80), pose angles(3), lighting parameters(27), translation(3). The first 257 dimension output is the same as the variable "coeff" in line 48 in demo.py. This coeff is then sent to an analytic image formation process to obtain a reconstructed image.

nwatab commented 4 years ago

@YuDeng Thank you for explanation. Now everything is clear 🎉