Open Lotayou opened 5 years ago
Hi,
I think you're almost there but off by just a small amount for not accounting for the crop/scale preprocessing or not projecting the points with the same projection function. The 2D joints that the model/demo returns is projected from the 3D joints and they align to the rendered mesh, so you can simply do the same thing, except project all of the vertices instead of just the joint3d points. This happens here: https://github.com/akanazawa/hmr/blob/5534c9b5791c3f5c9a5d4d3f0063d38fad065d81/src/RunModel.py#L119
This can be tricky, since the model operates on the preprocessed image (scaled and cropped). So first, make sure that you can project the vertices just like the joints2d (pred_kp
) there. Becareful bc this is originally in the normalized [-1, 1] coord system (this is where it comes back to the preprocessed image coordinate system https://github.com/akanazawa/hmr/blob/5534c9b5791c3f5c9a5d4d3f0063d38fad065d81/src/RunModel.py#L170).
Then all you have to do is undo the scale and cropping.
Best,
A
Hello, do you know how to use mpi_inf_3dhp_to_tfrecords.py to convert mpi_inf_3dhp dataset? I failed because the code use jpg as input but the dataset I downloaded is consisting of videos. Do I need to use ffmpeg and write code to convert avi to jpg?
Hi @akanazawa thanks for the work. I would also like to get aligned coordinates of all vertices (on 2D image plane), how can I achieve that?
I've tried a simple affine matching strategy but it won't work. The aligned verts deviate from the actual person, as shown below:![fashion](https://user-images.githubusercontent.com/33449901/63477166-7951b280-c4b6-11e9-927e-4507dfae47db.jpg)
Here are the codes I use for alignment, I first save the network output in a single pickle file.
Then I create an affine mapping to get joints3d onto joints2d:
Is there a quick workaround to get aligned vertex coordinates from renderer output? Thanks?