youngLBW / HRN

[CVPR2023] A Hierarchical Representation Network for Accurate and Detailed Face Reconstruction from In-The-Wild Images.
https://younglbw.github.io/HRN-homepage/
Apache License 2.0
452 stars 38 forks source link

Bad results when face has bangs #32

Open doublexxking opened 1 year ago

doublexxking commented 1 year ago

As title mentioned, if the face has bangs, especially girls, the high mesh will get very bad results on forehead. However, I know you got very high mark on forehead part in REALY. Why I got this situation and Do you have some suggestions to solve this problem?

The following pictures show some examples.

Screenshot from 2023-06-28 11-58-32

Screenshot from 2023-06-28 10-49-04

yosun commented 1 year ago

Do some basic image processing to remove hair

doublexxking commented 1 year ago

Although I use bareheaded, the forehead is weird sometimes, as shown following Screenshot from 2023-06-28 14-43-33 Screenshot from 2023-06-28 14-44-18

yosun commented 1 year ago

use some quick image processing techniques to remove the weird shine and uneven lighting in forehead

doublexxking commented 1 year ago

Thanks for your reply and I will try your method. By the way, is there some parameters could be set to control the weight of changes? For example, I do not want to forehead to change so much towards the mean face.

doublexxking commented 1 year ago

Moreover, I find the reason is not the weird shine or uneven lighting... The reason is the girl face is quite smaller than the mask. And the algorithm think the boundary of head is wrinkle as following shown. Screenshot from 2023-07-01 16-25-26

yosun commented 1 year ago

it seems you got the forehead to be smooth. you can just use dlib 16 face border points to adjust the mesh for smaller faces

kizum commented 1 year ago

I suppose the high mesh is not the results that author used to evaluate on the Realy or other benchmark. The displacement map should only effects the texture part by change the normal of the face. The geometry of the final results should only be Deep3DFace result + deformation map.

The high mesh is just an addition toy output of the HRN, as mentioned in #6 . If I was wrong, pls let me know. :)