Distance between face and camera

[Adra8373]

Hi

I have a question when I used frames that have face close to the camera the fitting process is failed. that means the generated 3D model is not same as the 2D frame, could you explain to me why that happens, please?

[patrikhuber]

Hi,

I don't know what you mean. Can you attach a picture?

[Adra8373]

I mean when the picture contains a face that close to the camera. for example the attached pic.

[headupinclouds]

You may want to include some indicator of what the results look like along with your initial landmarks and the functions+parameters you are using. I'm currently experimenting with eos as well. Here are the results I get out of the box for the landmarks shown in red:

This is using calls/parameters from the non-ceres based sample app:

        std::tie(mesh, rendering_params) = eos::fitting::fit_shape_and_pose(
            morphable_model,
            blendshapes,
            landmarks,
            landmark_mapper,
            image.cols,
            image.rows,
            edge_topology,
            ibug_contour,
            model_contour,
            50, boost::none, 30.0f);

FWIW, I did have to tinker with the landmarks to fit this sides of this particular face/pose well (completely unrelated to eos).

[Adra8373]

Thank you so much for replying. What program are you using to get the landmarks? I am using another software that does not include contour landmarks.

[headupinclouds]

Sure. I put a small eos sample together using the dlib face detector and shape_predictor class "One Millisecond Face Alignment with an Ensemble of Regression Trees" (Kazemi, et al)) w/ the pre-trained ibug-68 model.

See: https://github.com/headupinclouds/hunter_eos_example

This was initially a response to this issue https://github.com/patrikhuber/eos/issues/140#issuecomment-311852989.

This is using a fairly recent0.12.1tag of eos in the hunter package manager, which manages the dependencies (although eos is already pretty easy to use). I'm planning to keep this package reasonably up-to-date (I see there are a lot of updates in progress.)

Caveat: As I mentioned above, the internal face landmarks fit reasonably well w/ the stock model, but if you run the app as is, you will notice that the outer face contour is a little wide -- even with a fair amount of jittering. I fixed those landmarks by hand to keep the landmark initialization phase independent from eos fitting. Let me know if you have any issues.

[Adra8373]

Thanks. But as I said I do not have a contour landmarks, so there is no problem in initialization phase. What do you think the problem could you explain, please?

[headupinclouds]

Sorry, I'm still not quite sure what you mean. (I'm also just learning eos internals myself.) If you post a SSCCE example somewhere, it will be much easier for someone to help you.

[patrikhuber]

@Adra8373: If you do not use contour landmarks, then, naturally, it will not fit well around the face contour. Particularly if the face is very wide or narrow or has some perspective effect (eos uses a scaled orthographic camera model).

@headupinclouds Thank you very much for your replies! And great to see the hunter package up-to-date :-) And yep, I hope to bring some nice updates to the libraries over the next months.

[Adra8373]

Thank you both for your replays. Many thanks