search

cleardusk / 3DDFA

The PyTorch improved version of TPAMI 2017 paper: Face Alignment in Full Pose Range: A 3D Total Solution.
MIT License
3.61k stars 653 forks source link

Hi, I have a few questions about the Dlib (face detection and cropping) #102

Closed Jian-danai closed 4 years ago

Jian-danai commented 5 years ago

Hi, I found your repo really useful and your introduction is really clear enough but I still have a few questions about the Dlib (face detection and cropping), GPU computing, and the obj files.

  1. I found that the inference of the train data(train_aug_120x120.zip) often fails because of the failure of face detection and cropping. For example, I have run main.py for train_aug_120x120, only 22300/52752 faces can be detected to generate obj, depth, ...etc. Is there any advice for me to make the face detection more accurate?

A: Oh, I find that the comment in your code 'two-step for more accurate bbox to crop face' maybe works and improves the possibility of detecting and cropping faces? Also found some suggestions on https://github.com/cleardusk/3DDFA/issues/96 Sorry for the inconvenience.

  1. Is there a way to use multi-GPU for the inference process? (main.py)

  2. Is there a way to modify the accuracy of the obj（3D models）？I mean, can I just modify some parameters in your code to improve the accuracy. We can see the obj from Blender2.80 that its accuracy is not so satisfactory especially for the 'nose' but maybe it is because the RGB images are too rough. image

A: Found instructions on (https://github.com/cleardusk/3DDFA/issues/48), I need to refer to papers for high-fidelity face reconstruction, haha.

  1. tri = sio.loadmat('visualize/tri.mat')['tri'] Is the tri.mat(3D mesh triangle indices) still available for all color image in the dataset you provided?

A: Found instructions on (https://github.com/cleardusk/3DDFA/issues/66) Use tri.mat, not tri_refine.mat. The previous can be applied to any image or video.

  1. write_obj_with_colors, does the 'colors' mean the texture?

A: Found instructions on (https://github.com/YadiraF/PRNet/blob/fc12fe5e1f1462bdea52409b213d0cf1c8cf6c5b/utils/write.py#L16) and face3d.

Thanks

cleardusk commented 5 years ago
  1. The Dlib face detector is outdated for a long time. I have recommended some accurate and fast face detectors: FaceBoxes.PyTorch, libfacedetection, ZQCNN, described in readme.md. You can use them to substitute Dlib and welcome to propose a PR.
  2. Multi-GPU inference is easy, you can refer to the DataParallel function in Pytorch.
  3. I have updated a demo recently, you can dig into the demo code for more details.

Your questions are a little messy, but welcome for your interest in this repo.