search

sicxu / Deep3dPortrait

Deep 3D Portrait from a Single Image
MIT License
386 stars 72 forks source link

about preprocessing #15

Closed MontaEllis closed 3 years ago

MontaEllis commented 4 years ago

Hi, Your work is so great! But I wonder how to do the preprocessing step(landmark detecting, alignment...). How can I use my own image to do the forward step? Thanks a lot!

sicxu commented 4 years ago

Sorry for the late reply. For face alignment, we have described in README.md how to obtain 68 landmarks by using Bulat et al.'s method, some addtional steps should be applied for their results. For face segmentation, we use Lin et al.'s method in our paper. However, since they have not released training code and pretrained model, you can reimplement their method yourself. Another option is to use this repo for segmentation mask (Sorry for your inconvenience, we have not carefully checked this repo. I am quite busy at other projects right now, we may update some code to make it easy to use when I'm available).

brunovollmer commented 4 years ago

Which format is the segmentation file in? I used the alternative repo you suggested but I'm a bit stuck with the output of the face segmentation that is needed for your approach. Thanks in advance!

sicxu commented 4 years ago

The mask file defined in step 2 is .mat file. You can refer to example mask files for more details. https://github.com/sicxu/Deep3dPortrait/blob/fad83d78cea303eb3da24afd59fac960eb3f6f38/step2_face_segmentation.py#L18 The segmantic label we used is defined as follows. https://github.com/sicxu/Deep3dPortrait/blob/fad83d78cea303eb3da24afd59fac960eb3f6f38/utils/face_parsing.py#L10-L23