search

mks0601 / I2L-MeshNet_RELEASE

Official PyTorch implementation of "I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Image", ECCV 2020
MIT License
720 stars 128 forks source link

Was Freihand model trained with param regressor? Demo problem #15

Closed pablovela5620 closed 4 years ago

pablovela5620 commented 4 years ago

I'm trying to modify the demo to checkout the trained Freihand model but get the following error

RuntimeError: Error(s) in loading state_dict for DataParallel:
        Missing key(s) in state_dict: "module.param_regressor.fc.0.weight", "module.param_regressor.fc.0.bias", "module.param_regressor.fc.1.weight", "module.param_regressor.fc.1.bias", "module.param_regressor.fc.1.running_mean", "module.param_regressor.fc.1.running_var", "module.param_regressor.fc.3.weight", "module.param_regressor.fc.3.bias", "module.param_regressor.fc.4.weight", "module.param_regressor.fc.4.bias", "module.param_regressor.fc.4.running_mean", "module.param_regressor.fc.4.running_var", "module.param_regressor.fc_pose.0.weight", "module.param_regressor.fc_pose.0.bias", "module.param_regressor.fc_shape.0.weight", "module.param_regressor.fc_shape.0.bias", "module.human_model_layer.th_betas", "module.human_model_layer.th_shapedirs", "module.human_model_layer.th_posedirs", "module.human_model_layer.th_v_template", "module.human_model_layer.th_J_regressor", "module.human_model_layer.th_weights", "module.human_model_layer.th_faces".

I downloaded pretrained snapshot_24 and it looks like its missing all of the param regressor weights. Something I'm doing wrong?

pablovela5620 commented 4 years ago

looks like a current work around is to switch strict to False

model.load_state_dict(ckpt['network'], strict=False)

and remove all param outputs

mks0601 commented 4 years ago

Yes. FreiHAND model is trained only for the lixel stage, and I did not additionally trained for the param stage. For the demo, I changed the code to include , strict=False). I guess it will work now? Or you can follow the README instructions for the full code, which works.

pablovela5620 commented 4 years ago

For sure! Demo worked great for full body pose estimation via SMPL, I wanted to see how it worked with MANO using a hand bounding box which is why I was asking about the param stage. Thank you for the help

hungsing92 commented 3 years ago

@pablovela5620 Can you get a good result of the freiHand model?

pablovela5620 commented 3 years ago

Yes, I was surprised as to how well the model was able to generalize. You just need to make sure you have the correct bounding box as well as a reasonable value for the root node, I had to make a few modifications to get it to work since the original demo is focused on SMPL estimation rather than MANO @hungsing92