patrikhuber / 4dface

Real-time 3D face tracking and reconstruction from 2D video
https://www.4dface.io
Apache License 2.0
786 stars 248 forks source link

Face Fine Details Fitting #30

Closed DannyJieweiLi closed 7 years ago

DannyJieweiLi commented 7 years ago

I noticed that the fitting module of eos only contains code of fitting landmarks (related to function fitting::fit_shape_and_pose). The result 3D model is smooth and do not have much details. Is there any way to improve the details (i.e wrinkles) of the result model? I thought that the detail missing is caused by the low resolution of morphable model (3448 vertices model). But even I used the higher resolution model, I do not get the expected details.

patrikhuber commented 7 years ago

Hi Danny,

Thanks for your comment!

Yes, all your observations are correct. The high resolution model would not help much since it's doing landmarks-fitting. Here's two things:

1) You can build something on top of eos - i.e. use the result of the landmark fitting to do some more precise fitting, e.g. analysis-by-synthesis, Shape-from-Shading, or something like that. There's a ton of literature about it.

2) I have plans to improve the fitting of eos, and will work on that over the next months. However there will nothing be ready soon, and it will definitely take a few months, and I cannot promise any results at this point.

DannyJieweiLi commented 7 years ago

Thanks your reply, Patrick. I will look into the detail keeping algorithms. ^_^

sunjunlishi commented 7 years ago

great,author!

patrikhuber commented 7 years ago

Great, I will close the issue in the mean time. I will update it when I make some improvements to the fitting algorithm. It's also worth following the releases of eos. Also feel free to update this issue if you implement something! :-) Thanks!