search

MartaYang / KITRO

Codes for the CVPR 2024 paper: "KITRO: Refining Human Mesh by 2D Clues and Kinematic-tree Rotation"
MIT License
42 stars 1 forks source link

How can we obtain J_regressor for custom dataset? #5

Open chey0313 opened 2 months ago

chey0313 commented 2 months ago

Hi, thanks for your exciting work. I am curious about J regressor. Actually, I want to train the j_regressor on a custom dataset.

Can you tell me how the "/data/extradata/spin/J_regressor_h36m.npy" was obtained from? Is it mentioned in the paper?

Thanks. Ilchae Jung

chey0313 commented 2 months ago

Additionally, I am curious about how the file, "data/ProcessedData_CLIFFpred_w2DKP_3dpw.pt," was obtained. Could you provide the code at your earliest convenience?

Thanks, Ilchae Jung

MartaYang commented 2 months ago

Hi Ilchae,

Thank you for your interest in our work!

The J_regressor was originally proposed in SMPL. It's a matrix that maps mesh vertices to 3D joint locations, aiming to minimize the vertex error of the model on a given training set.

In our case, we didn't train the J_regressor ourselves but directly used it as provided in previous works like SPIN. As mentioned in the SMPL paper, a common approach to compute the J_regressor is by using non-negative least squares, which ensures that the resulting joint locations are a linear combination of the mesh vertices with non-negative weights.

You can refer to the SMPL paper for more detailed insights on the regression strategy.

g8392 commented 1 month ago

Hi Ilchae,

Thank you for your interest in our work!

The J_regressor was originally proposed in SMPL. It's a matrix that maps mesh vertices to 3D joint locations, aiming to minimize the vertex error of the model on a given training set.

In our case, we didn't train the J_regressor ourselves but directly used it as provided in previous works like SPIN. As mentioned in the SMPL paper, a common approach to compute the J_regressor is by using non-negative least squares, which ensures that the resulting joint locations are a linear combination of the mesh vertices with non-negative weights.

You can refer to the SMPL paper for more detailed insights on the regression strategy.

Thanks for the project Marta, any plans on releasing a demo for us mortals to be able to use in out custom media and export for 3d?