search

zju3dv / EasyMocap

Make human motion capture easier.
Other
3.65k stars 450 forks source link

Mesh and Detection point mismatch (slight mismatch acceptable?) #331

Open Dipankar1997161 opened 1 year ago

Dipankar1997161 commented 1 year ago

Hello @chingswy , Hope you doing well.

So apparently, I was able to project human3.6m groundtruth and got the smpl reconstructed. However, there is a slight mismatch in the 4 cameras' output. Camera 01 and 03 (Back camera detection is more accurate and Camera 4 is the worst of all) The same joint (nose and head, appear at two separate sides on the right side of images)

lets say: The acutal R matrix is 

  'R':[array([[-0.91536173,  0.40180837,  0.02574754],
         [ 0.05154812,  0.18037357, -0.98224649],
         [-0.39931903, -0.89778361, -0.18581953]])

  I used this: 

  data: [-0.915, 0.402, 0.026, 0.052, 0.180, -0.982, -0.399, -0.898, -0.186]

000000

000000_smpl

Even though I gave the correct joint coordinate, the 4th camera image, Thorax/Neck joint is incorrect while it is correct in the rest of the images.

Could this be a possible error in the Camera Parameter values? If the coordinate values were wrong, then it should be wrong in all of them. But the 4th camera output is yielding a slight mismatch in comparison with the rest.

Can you clarify if this is the case? if any other let me know. Or this slight mismatch is acceptable?

chingswy commented 1 year ago

Is this error originated from the process of acquiring SMPL? This type of error has a minor impact on neural body.

Dipankar1997161 commented 1 year ago

Is this error originated from the process of acquiring SMPL? This type of error has a minor impact on neural body.

it was a slight detection error, but its major in smpl, the previous images was for the 'repro' and 'smplmesh'. Upon checking the SMPL-repro folder, I found the 4th camera image, nose joints totally off, while the other 3 images are acceptable.

000001

I passed The human3.6m ground-truth joints "Spine, Neck and Head" to the body25 model. And I believe body25 requires "Nose" parameters to draw the eyes and ears.

So I think this was the closest detection possible for human3.6m fitting directly on body25 using GroundTruth.

Do you think there is a "Nose" joint in Human3.6m? I passed original joint index 14, 13 of h36m as 0, 1 for body25. Like manual fitting. The camera values I acquired from his repo (scripts/dataset/pre_h36m_camera.py)

If there's anything wrong you thing, let me know