search

YuvalNirkin / face_segmentation

Deep face segmentation in extremely hard conditions
Apache License 2.0
730 stars 151 forks source link

Unable to segment eye-glasses #25

Closed HoiM closed 3 years ago

HoiM commented 3 years ago

Hello Yuval,

Thank you for your project. I am also working on face swap projects. Now I am trying to tackle the problem of face occlusions.

I find the performance on the README page really amazing. However, after installation I find that the performance on my face-swap training images not as good as your demo image. The model is unable to segment eye-glasses.

Here are some examples:

manmankan_big_qipu0_202393805_0 86573937186_5b56efa1b302c33404bda177 manmankan_big_qipu0_215746105_0 785859323308_5b56ee91ef924e6e48d046bc manmankan_big_qipu0_215790205_0 719922288978_5b56fcf9b302c3340590677d

I ran the model by this command:

face_seg_image my_image.jpg -o . -m ../data/face_seg_fcn8s.caffemodel -d ../data/face_seg_fcn8s_deploy.prototxt

where my_image.jpg is a 256x256 image in which the face is aligned via 5 landmarks (eyes, nose and mouth).

Do you have any ideas on this? Or is there anything wrong with my input format?

Thank you in advance.

YuvalNirkin commented 3 years ago

The model was trained to remove the segmentation of only a limited number of sunglasses and so might fail sometimes. For glasses (with clear glass), the model was trained to include them as part of the face.