search

tranluan / Nonlinear_Face_3DMM

Source code for "Nonlinear 3D Face Morphable Model"
http://cvlab.cse.msu.edu/project-nonlinear-3dmm.html
Apache License 2.0
671 stars 123 forks source link

Pytorch reimplementation #37

Open zhangmozhe opened 4 years ago

zhangmozhe commented 4 years ago

Hi, I really like this work. I just wonder whether you have the plan to reimplement it using Pytorch? Or release the pytorch pre-trained model?

HoiM commented 4 years ago

I am currently working on this project with PyTorch and PyTorch3D. Things are almost done except the settings of the camera so that the 8 transformation parameters can work with the rendered image. Now I hope the author could provide the parameters of the camera.

invisprints commented 3 years ago

@HoiM Hello, any progress？

HoiM commented 3 years ago

@HoiM Hello, any progress？

Got no reply from the author...

invisprints commented 3 years ago

@HoiM I am confused about the parameters of the camera you said. are they different from camera projection parameters？In the paper, it said "the camera projection parameters estimated by the encoder"

chaoshiedwin commented 3 years ago

I am currently working on this project with PyTorch and PyTorch3D. Things are almost done except the settings of the camera so that the 8 transformation parameters can work with the rendered image. Now I hope the author could provide the parameters of the camera.

Hi, did you find that the z-buffer kernel is not differentiable in the code? How can you resovle this problem with pytorch?

HoiM commented 3 years ago

@invisprints Maybe I was confused. The camera model used in the paper is weak perspective projection camera model. Based on my understanding, the 8 parameters can directly convert a 3D point to a plane. While in my implementation, I used the perspective projection camera model, where field of view, location of the camera in the world system, and the focal length must be set. Maybe the 8 parameters are already enough.

Please feel free to correct me if anything I said was wrong.

HoiM commented 3 years ago

@chaoshiedwin I switched to another work and successfully implemented the rendering process. Here the camera model is weak perspective model. But I used OpenGLPerspectiveCamera in PyTorch3D, where Z-buffer is not needed to be coded by hand.

Moreover, based on my understanding, z-buffer is more of like a mask where back-propagation is not needed.