search

patrikhuber / eos

A lightweight 3D Morphable Face Model library in modern C++
Apache License 2.0
1.89k stars 596 forks source link

Fitting 3D Landmarks to mesh #280

Closed jmhummel closed 4 years ago

jmhummel commented 4 years ago

I'm using the 3D-FAN network to generate 3D ibug landmarks, and I wanted to fit a mesh to these points.

I've been able to use just the 2D component for each landmark to generate the mesh, but I'm interested if there's a way to fit the mesh on the 3D landmarks

jmhummel commented 4 years ago

Just to clarify, these landmarks are not true 3D, they are an orthographic projection

patrikhuber commented 4 years ago

Sorry for the late reply. Yes, you should absolutely be able to do that. You can just adapt the linear system of equations accordingly in the shape fitting (and in the camera fitting possibly too). I recommend that you read O. Aldrian & W. Smith, Inverse Rendering of Faces with a 3D Morphable Model, PAMI 2013 to get familiar with the linear system of equations. And then read G. Hu et al., Efficient 3D Morphable Face Model Fitting, PR 2017, who do linear fitting in 3D - it should be more or less exactly what you're asking for.

I'm going to close this as it's a research question and not an issue with the eos library. If you're working on this (or have been working), it would be nice to hear back how it worked out, and if you have a PR or something that you'd like to contribute, feel free to always re-open the issue. Thanks!