search

StoryMY / take-off-eyeglasses

Official pytorch implementation of paper "Portrait Eyeglasses and Shadow Removal by Leveraging 3D Synthetic Data" (CVPR 2022).
130 stars 17 forks source link

Does pretrained model can segment the lens of eyeglasses too? #10

Open phuongpropk05 opened 2 weeks ago

phuongpropk05 commented 2 weeks ago

I want to remove the léns of sunglasses too, how can i do that. I hope you can help me soon

StoryMY commented 2 weeks ago

Our work is not good at removing the lenses of eyeglasses (see Limitations section in our paper for more details).

One way to compel the networks is to replace our glass mask with the complete glass mask (e.g., detected by face-parsing), which includes the lenses region. However, I guess the final result would be really bad as it is difficult to recover eye region under such serious occlusion.

I recommend to use generative models to solve this problem, but I am not sure which work is the SOTA. Maybe you can conduct some research on GAN and Diffussion Model. In my impression, many related works are good at inpainting task. I hope these works can help you.