Fitting is bad when some joints are not visible

[karfly]

Hi, @vchoutas! First of all, thank you for such nice work and clear code!

I want to fit SMPL-X model to images, where some joints are not visible, e.g. upper-body photos or even face photos. I tried several images and found out that the method works much worse when some joints are out of the frame (you can see examples below). The resulting fits don't much the input at all.

I believe this issue can be somehow fixed by nullifying some joints' weights during optimization or by some other optimization tweak. What do you think is the best way to do it?

Input	SMPLify-X

[vchoutas]

If you know that people are standing, or sitting, you could disable the optimization for the global rotation of the body, as well as increasing the body pose regularization for the final stages.

[youwei-kafuu]

Hi! @karfly I'm recently reserching on upper body reconstruction.Do you have any ideas about reconstruction of upper body by using SMPL？ thanks in advance.

[rongakowang]

Hi @vchoutas:

Could you please provide more detailed instructions on how to disable optimization for global rotation and increase body pose regularization? I have a set of images that contain only the upper part of the body but the fitting result is really bad. I have checked openpose result and it seems that all the leg part key points are 0. Will this affect the performance? Thanks in advance

[vchoutas]

@WangRong089 you could disable global rotation by setting requires_grad=False. To increase body pose regularization you have to increase the weights for the corresponding term. If the hips are missing then translation will be wrongly estimated. Right now the code does not support any other initialization methods.

[Usama-Tahir]

Can you be more specific as to which requires_grad needs to be false? I have set camera rotation requires_grad false but I don't believe it is the same as global body rotation.

Also did anyone get accurate results after following the mentioned solution?

[vchoutas]

Camera rotation is not the same as global body rotation, the latter has a separate variable.

[stephstoll91]

I might be a bit late to the party, but in case anyone is still struggling with this: like suggested by @vchoutas I set the body_model's global_orient requires grad to False like this: body_model.global_orient.requires_grad = False

That fixed the problem for me. I am working on upper body only and had the same issues as @karfly.

[gulzainali98]

are you getting upside down orientation after making changes?

[vchoutas]

You could also use ExPose work to initialize the translation from the camera and avoid problems when the person is cropped.

[nerdygirl21]

body_model.global_orient.requires_grad

Hi, can anyone share which file and line to make these changes in?