search

wangsen1312 / joints2smpl

fit smpl parameters model using 3D joints
166 stars 11 forks source link

Can the project add a CNN and train it to fit out body pose ? #12

Open keylLi opened 7 months ago

keylLi commented 7 months ago

In your code, the optimizer is used directly to optimize the body pose, etc., so it is not possible to model the relationship between joint3d and body pose. I'm wondering if it's possible to add a CNN for training and to save the parameters for easy deployment in the project.

wangsen1312 commented 5 months ago

@keylLi For the configuration, what's the input? image or 3D joints?

keylLi commented 5 months ago

@wangsen1312 My inputs are keypoints and following your code I can get a nice smpl mesh, but the parameter optimization process is slow, about 1 frame per second. Is this speed normal? Finally, I was wondering if I can use neural network models (CNN etc), instead of the slow parameter optimization process.

wangsen1312 commented 5 months ago

@keylLi For the optimization, 1 fps/second is normal, I did not do any extra optimization on the code. For the end-to-end network, I think you can get some results. You can also use the network results as the initial and reduce the optimization iteration to improve the overall speed.