Easy to fit to a strange shape with 2d landmark.

[Guocode]

When I try to fit my 2d landmark, I find it's easy to get a extremely distorted shape. How can I keep the a normal shape as well as minimize the fitted 2d projection feature loss?

[TimoBolkart]

For correct landmarks this should not happen. Maybe check your landmark labels that they correspond with the FLAME landmarks. Once you are sure that everything is correct with the landmarks and you still get extremely distorted shapes, you can choose a higher weight in the code for the different priors (i.e. shape, pose, expression). Increasing these prior weights will regularize the parameters more and therefore constrain the shapes that you will get. Does this help you?

[Guocode]

@TimoBolkart In fact, in large pose or some cases that is difficult to annotate, there is a gap between the annotated landmarks and the flame-defined landmark, so I think maybe the loss function should be modified to overcome the gap，for example using exponent function instead of l1 when loss is small, so that it will be tolerant for the annotation deviation. I have tried higher weight for priors you mention, which will lead to a more stable result but larger fitted landmark projection distance. The better solution is always adding more constraint like Ringnet.

[TimoBolkart]

You are absolutely right in cases of wrong or noisy landmarks, one could use some robustifier function before applying the loss, if some keypoints are not well fitted, one could either use a different loss (e.g. some exponential function) or weight some landmarks higher before summing the losses up. Other constraints like multi-person or multi-view information of course can also help to get better results.

[TimoBolkart]

Did you manage to solve your problem?