search

AaronJackson / vrn

:man: Code for "Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression"
http://aaronsplace.co.uk/papers/jackson2017recon/
MIT License
4.52k stars 742 forks source link

voxelization #18

Closed DontStopMeNow closed 6 years ago

DontStopMeNow commented 6 years ago

Hi, could you post the voxelization code? Or tell more about the used algorithm. I tried some tools but they generate a volume model with artifacts like this: image

AaronJackson commented 6 years ago

Nearly there it seems. :) Check Figure 2 in the paper - we rotate to frontal first using the 3DDFA pose parameters and then rotate the entire 3D volume back so that spatial correspondence is preserved. If you can fill holes in advance, such as the ones on the nose and mouth, you can probably use something like this, but it will be slower than filling from the front. I am sorry but we have decided not to release any of the code required for training just yet.

Jim61C commented 6 years ago

Hi, I just have one question on the coordinate frame that the voxelisation is performed, do you perform it on the original image frame (arbitrary width and height) or do you perform the voxelisation in the coordinate frame of 192x192 image (which means that you scale and translate the mesh)? Thanks!

AaronJackson commented 6 years ago

All faces are normalised using a landmark detector to fit into the 192x192 space. The same transformation is always applied to the mesh, hence, the voxelisation is done in the image space.

Jim61C commented 6 years ago

Hi!

Thanks very much for the clarification! I just have another question is that I can see that during inference, you are padding the image with white background. During training, are you doing the same or padding with black or using a different padding mechanism? As all the images from 300W-LP are already synthesized with a black background. Thanks!

AaronJackson commented 6 years ago

I don't remember sorry. I think I used black backgrounds during training, and the MATLAB version of this code should also be using black backgrounds. The python version might be doing something else. :man_shrugging:

Jim61C commented 6 years ago

Cool! Thx for the clarification! I am just having another question is that would you mind providing a rough estimate on how long it would take to train the model from scratch using 300W-LP? Thanks.

AaronJackson commented 6 years ago

Training is usually a couple of days on a pair of Titan X or 1080 TIs. The part that takes the longest is voxelising your dataset.

Jim61C commented 6 years ago

Cool! Thanks for the clarification! So you are using multi-GPU training?