search

patrikhuber / eos

A lightweight 3D Morphable Face Model library in modern C++
Apache License 2.0
1.91k stars 597 forks source link

About blendshape fitting #71

Closed LCLL closed 7 years ago

LCLL commented 7 years ago

@patrikhuber I want to know if the coefficients achieved from the function "fit_blendshapes_to_landmarks_linear" are in range [0..1] ? Thanks.

patrikhuber commented 7 years ago

We're using standard linear least squares in this function, so no, the coefficients are not constrained. Internally I'm actually using a NNLS solver (non-negative LS) for this function (which is the correct way to go since the blendshapes are linear (positive) offsets). It's not my solver, but I might be able to share it in the future. However, in practice, in a few tests I've done, the results looked pretty similar.

Do you have any issue with it, or why the question?

LCLL commented 7 years ago

Thank you for your quick reply. I just use Facial Action Coding System to construct the blendshapes so that the coefficients need to be set in range [0...1]. I use projection-based gradient descent to get the coefficients and want to find is there any more efficient way to solve it. Do you have any idea about it ?

patrikhuber commented 7 years ago

As I mentioned (and I think this applies to your case too), NNLS is what you want.

Can you share your FACS blendshapes? How did you construct them and from what data?

LCLL commented 7 years ago

Yes, I just found that the Ceres provides NNLS solvers (http://ceres-solver.org/nnls_tutorial.html) that is what I need. I will try to compare it with the projection based gradient descent methods. Thank you.

patrikhuber commented 7 years ago

Yes, I just found that the Ceres provides NNLS solvers (http://ceres-solver.org/nnls_tutorial.html) that is what I need.

This is actually not NNLS. NNLS is non-negative least squares. What you linked to is non-linear least squares. Big difference. I don't know though why the Ceres html filename is "nnls_tutorial.html", there's nothing about "nnls" there :-)

You can still use Ceres though, I guess if you use a non-linear solver for a linear problem, it'll just converge in the first iteration. (or maybe your cost function is nonlinear anyway.)

patrikhuber commented 7 years ago

You can use the function fit_blendshapes_to_landmarks_nnls(...) now. Make sure to run git submodule update --init after pulling, to get the required submodule.