search

dulucas / siMLPe

A Simple Baseline for Human Motion Prediction
MIT License
134 stars 20 forks source link

Missing key(s) in state_dict while loading the pre-trained h36m model #9

Closed bhomaidan1990 closed 6 months ago

bhomaidan1990 commented 1 year ago

Following the Readme about Evaluation: python test.py --model-pth /home/user/git/siMLPe/checkpoints/h36m_model.pth When loading the pre-trained model I get an error: Missing key(s) in state_dict

model.load_state_dict(state_dict, strict=True)
  File "/home/user/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 2041, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for siMLPe:
        Missing key(s) in state_dict: "motion_mlp.mlps.0.fc0.fc.weight", "motion_mlp.mlps.0.fc0.fc.bias", "motion_mlp.mlps.0.norm0.alpha", "motion_mlp.mlps.0.norm0.beta", "motion_mlp.mlps.1.fc0.fc.weight", "motion_mlp.mlps.1.fc0.fc.bias", "motion_mlp.mlps.1.norm0.alpha", "motion_mlp.mlps.1.norm0.beta", "motion_mlp.mlps.2.fc0.fc.weight", "motion_mlp.mlps.2.fc0.fc.bias", "motion_mlp.mlps.2.norm0.alpha", "motion_mlp.mlps.2.norm0.beta", "motion_mlp.mlps.3.fc0.fc.weight", "motion_mlp.mlps.3.fc0.fc.bias", "motion_mlp.mlps.3.norm0.alpha", "motion_mlp.mlps.3.norm0.beta", "motion_mlp.mlps.4.fc0.fc.weight", "motion_mlp.mlps.4.fc0.fc.bias", "motion_mlp.mlps.4.norm0.alpha", "motion_mlp.mlps.4.norm0.beta", "motion_mlp.mlps.5.fc0.fc.weight", "motion_mlp.mlps.5.fc0.fc.bias", "motion_mlp.mlps.5.norm0.alpha", "motion_mlp.mlps.5.norm0.beta", "motion_mlp.mlps.6.fc0.fc.weight", "motion_mlp.mlps.6.fc0.fc.bias", "motion_mlp.mlps.6.norm0.alpha", "motion_mlp.mlps.6.norm0.beta", "motion_mlp.mlps.7.fc0.fc.weight", "motion_mlp.mlps.7.fc0.fc.bias", "motion_mlp.mlps.7.norm0.alpha", "motion_mlp.mlps.7.norm0.beta", "motion_mlp.mlps.8.fc0.fc.weight", "motion_mlp.mlps.8.fc0.fc.bias", "motion_mlp.mlps.8.norm0.alpha", "motion_mlp.mlps.8.norm0.beta", "motion_mlp.mlps.9.fc0.fc.weight", "motion_mlp.mlps.9.fc0.fc.bias", "motion_mlp.mlps.9.norm0.alpha", "motion_mlp.mlps.9.norm0.beta", "motion_mlp.mlps.10.fc0.fc.weight", "motion_mlp.mlps.10.fc0.fc.bias", "motion_mlp.mlps.10.norm0.alpha", "motion_mlp.mlps.10.norm0.beta", "motion_mlp.mlps.11.fc0.fc.weight", "motion_mlp.mlps.11.fc0.fc.bias", "motion_mlp.mlps.11.norm0.alpha", "motion_mlp.mlps.11.norm0.beta", "motion_mlp.mlps.12.fc0.fc.weight", "motion_mlp.mlps.12.fc0.fc.bias", "motion_mlp.mlps.12.norm0.alpha", "motion_mlp.mlps.12.norm0.beta", "motion_mlp.mlps.13.fc0.fc.weight", "motion_mlp.mlps.13.fc0.fc.bias", "motion_mlp.mlps.13.norm0.alpha", "motion_mlp.mlps.13.norm0.beta", "motion_mlp.mlps.14.fc0.fc.weight", "motion_mlp.mlps.14.fc0.fc.bias", "motion_mlp.mlps.14.norm0.alpha", "motion_mlp.mlps.14.norm0.beta", "motion_mlp.mlps.15.fc0.fc.weight", "motion_mlp.mlps.15.fc0.fc.bias", "motion_mlp.mlps.15.norm0.alpha", "motion_mlp.mlps.15.norm0.beta", "motion_mlp.mlps.16.fc0.fc.weight", "motion_mlp.mlps.16.fc0.fc.bias", "motion_mlp.mlps.16.norm0.alpha", "motion_mlp.mlps.16.norm0.beta", "motion_mlp.mlps.17.fc0.fc.weight", "motion_mlp.mlps.17.fc0.fc.bias", "motion_mlp.mlps.17.norm0.alpha", "motion_mlp.mlps.17.norm0.beta", "motion_mlp.mlps.18.fc0.fc.weight", "motion_mlp.mlps.18.fc0.fc.bias", "motion_mlp.mlps.18.norm0.alpha", "motion_mlp.mlps.18.norm0.beta", "motion_mlp.mlps.19.fc0.fc.weight", "motion_mlp.mlps.19.fc0.fc.bias", "motion_mlp.mlps.19.norm0.alpha", "motion_mlp.mlps.19.norm0.beta", "motion_mlp.mlps.20.fc0.fc.weight", "motion_mlp.mlps.20.fc0.fc.bias", "motion_mlp.mlps.20.norm0.alpha", "motion_mlp.mlps.20.norm0.beta", "motion_mlp.mlps.21.fc0.fc.weight", "motion_mlp.mlps.21.fc0.fc.bias", "motion_mlp.mlps.21.norm0.alpha", "motion_mlp.mlps.21.norm0.beta", "motion_mlp.mlps.22.fc0.fc.weight", "motion_mlp.mlps.22.fc0.fc.bias", "motion_mlp.mlps.22.norm0.alpha", "motion_mlp.mlps.22.norm0.beta", "motion_mlp.mlps.23.fc0.fc.weight", "motion_mlp.mlps.23.fc0.fc.bias", "motion_mlp.mlps.23.norm0.alpha", "motion_mlp.mlps.23.norm0.beta", "motion_mlp.mlps.24.fc0.fc.weight", "motion_mlp.mlps.24.fc0.fc.bias", "motion_mlp.mlps.24.norm0.alpha", "motion_mlp.mlps.24.norm0.beta", "motion_mlp.mlps.25.fc0.fc.weight", "motion_mlp.mlps.25.fc0.fc.bias", "motion_mlp.mlps.25.norm0.alpha", "motion_mlp.mlps.25.norm0.beta", "motion_mlp.mlps.26.fc0.fc.weight", "motion_mlp.mlps.26.fc0.fc.bias", "motion_mlp.mlps.26.norm0.alpha", "motion_mlp.mlps.26.norm0.beta", "motion_mlp.mlps.27.fc0.fc.weight", "motion_mlp.mlps.27.fc0.fc.bias", "motion_mlp.mlps.27.norm0.alpha", "motion_mlp.mlps.27.norm0.beta", "motion_mlp.mlps.28.fc0.fc.weight", "motion_mlp.mlps.28.fc0.fc.bias", "motion_mlp.mlps.28.norm0.alpha", "motion_mlp.mlps.28.norm0.beta", "motion_mlp.mlps.29.fc0.fc.weight", "motion_mlp.mlps.29.fc0.fc.bias", "motion_mlp.mlps.29.norm0.alpha", "motion_mlp.mlps.29.norm0.beta", "motion_mlp.mlps.30.fc0.fc.weight", "motion_mlp.mlps.30.fc0.fc.bias", "motion_mlp.mlps.30.norm0.alpha", "motion_mlp.mlps.30.norm0.beta", "motion_mlp.mlps.31.fc0.fc.weight", "motion_mlp.mlps.31.fc0.fc.bias", "motion_mlp.mlps.31.norm0.alpha", "motion_mlp.mlps.31.norm0.beta", "motion_mlp.mlps.32.fc0.fc.weight", "motion_mlp.mlps.32.fc0.fc.bias", "motion_mlp.mlps.32.norm0.alpha", "motion_mlp.mlps.32.norm0.beta", "motion_mlp.mlps.33.fc0.fc.weight", "motion_mlp.mlps.33.fc0.fc.bias", "motion_mlp.mlps.33.norm0.alpha", "motion_mlp.mlps.33.norm0.beta", "motion_mlp.mlps.34.fc0.fc.weight", "motion_mlp.mlps.34.fc0.fc.bias", "motion_mlp.mlps.34.norm0.alpha", "motion_mlp.mlps.34.norm0.beta", "motion_mlp.mlps.35.fc0.fc.weight", "motion_mlp.mlps.35.fc0.fc.bias", "motion_mlp.mlps.35.norm0.alpha", "motion_mlp.mlps.35.norm0.beta", "motion_mlp.mlps.36.fc0.fc.weight", "motion_mlp.mlps.36.fc0.fc.bias", "motion_mlp.mlps.36.norm0.alpha", "motion_mlp.mlps.36.norm0.beta", "motion_mlp.mlps.37.fc0.fc.weight", "motion_mlp.mlps.37.fc0.fc.bias", "motion_mlp.mlps.37.norm0.alpha", "motion_mlp.mlps.37.norm0.beta", "motion_mlp.mlps.38.fc0.fc.weight", "motion_mlp.mlps.38.fc0.fc.bias", "motion_mlp.mlps.38.norm0.alpha", "motion_mlp.mlps.38.norm0.beta", "motion_mlp.mlps.39.fc0.fc.weight", "motion_mlp.mlps.39.fc0.fc.bias", "motion_mlp.mlps.39.norm0.alpha", "motion_mlp.mlps.39.norm0.beta", "motion_mlp.mlps.40.fc0.fc.weight", "motion_mlp.mlps.40.fc0.fc.bias", "motion_mlp.mlps.40.norm0.alpha", "motion_mlp.mlps.40.norm0.beta", "motion_mlp.mlps.41.fc0.fc.weight", "motion_mlp.mlps.41.fc0.fc.bias", "motion_mlp.mlps.41.norm0.alpha", "motion_mlp.mlps.41.norm0.beta", "motion_mlp.mlps.42.fc0.fc.weight", "motion_mlp.mlps.42.fc0.fc.bias", "motion_mlp.mlps.42.norm0.alpha", "motion_mlp.mlps.42.norm0.beta", "motion_mlp.mlps.43.fc0.fc.weight", "motion_mlp.mlps.43.fc0.fc.bias", "motion_mlp.mlps.43.norm0.alpha", "motion_mlp.mlps.43.norm0.beta", "motion_mlp.mlps.44.fc0.fc.weight", "motion_mlp.mlps.44.fc0.fc.bias", "motion_mlp.mlps.44.norm0.alpha", "motion_mlp.mlps.44.norm0.beta", "motion_mlp.mlps.45.fc0.fc.weight", "motion_mlp.mlps.45.fc0.fc.bias", "motion_mlp.mlps.45.norm0.alpha", "motion_mlp.mlps.45.norm0.beta", "motion_mlp.mlps.46.fc0.fc.weight", "motion_mlp.mlps.46.fc0.fc.bias", "motion_mlp.mlps.46.norm0.alpha", "motion_mlp.mlps.46.norm0.beta", "motion_mlp.mlps.47.fc0.fc.weight", "motion_mlp.mlps.47.fc0.fc.bias", "motion_mlp.mlps.47.norm0.alpha", "motion_mlp.mlps.47.norm0.beta". 
        Unexpected key(s) in state_dict: "motion_transformer.transformer.0.fc0.fc.weight", "motion_transformer.transformer.0.fc0.fc.bias", "motion_transformer.transformer.0.norm0.alpha", "motion_transformer.transformer.0.norm0.beta", "motion_transformer.transformer.1.fc0.fc.weight", "motion_transformer.transformer.1.fc0.fc.bias", "motion_transformer.transformer.1.norm0.alpha", "motion_transformer.transformer.1.norm0.beta", "motion_transformer.transformer.2.fc0.fc.weight", "motion_transformer.transformer.2.fc0.fc.bias", "motion_transformer.transformer.2.norm0.alpha", "motion_transformer.transformer.2.norm0.beta", "motion_transformer.transformer.3.fc0.fc.weight", "motion_transformer.transformer.3.fc0.fc.bias", "motion_transformer.transformer.3.norm0.alpha", "motion_transformer.transformer.3.norm0.beta", "motion_transformer.transformer.4.fc0.fc.weight", "motion_transformer.transformer.4.fc0.fc.bias", "motion_transformer.transformer.4.norm0.alpha", "motion_transformer.transformer.4.norm0.beta", "motion_transformer.transformer.5.fc0.fc.weight", "motion_transformer.transformer.5.fc0.fc.bias", "motion_transformer.transformer.5.norm0.alpha", "motion_transformer.transformer.5.norm0.beta", "motion_transformer.transformer.6.fc0.fc.weight", "motion_transformer.transformer.6.fc0.fc.bias", "motion_transformer.transformer.6.norm0.alpha", "motion_transformer.transformer.6.norm0.beta", "motion_transformer.transformer.7.fc0.fc.weight", "motion_transformer.transformer.7.fc0.fc.bias", "motion_transformer.transformer.7.norm0.alpha", "motion_transformer.transformer.7.norm0.beta", "motion_transformer.transformer.8.fc0.fc.weight", "motion_transformer.transformer.8.fc0.fc.bias", "motion_transformer.transformer.8.norm0.alpha", "motion_transformer.transformer.8.norm0.beta", "motion_transformer.transformer.9.fc0.fc.weight", "motion_transformer.transformer.9.fc0.fc.bias", "motion_transformer.transformer.9.norm0.alpha", "motion_transformer.transformer.9.norm0.beta", "motion_transformer.transformer.10.fc0.fc.weight", "motion_transformer.transformer.10.fc0.fc.bias", "motion_transformer.transformer.10.norm0.alpha", "motion_transformer.transformer.10.norm0.beta", "motion_transformer.transformer.11.fc0.fc.weight", "motion_transformer.transformer.11.fc0.fc.bias", "motion_transformer.transformer.11.norm0.alpha", "motion_transformer.transformer.11.norm0.beta", "motion_transformer.transformer.12.fc0.fc.weight", "motion_transformer.transformer.12.fc0.fc.bias", "motion_transformer.transformer.12.norm0.alpha", "motion_transformer.transformer.12.norm0.beta", "motion_transformer.transformer.13.fc0.fc.weight", "motion_transformer.transformer.13.fc0.fc.bias", "motion_transformer.transformer.13.norm0.alpha", "motion_transformer.transformer.13.norm0.beta", "motion_transformer.transformer.14.fc0.fc.weight", "motion_transformer.transformer.14.fc0.fc.bias", "motion_transformer.transformer.14.norm0.alpha", "motion_transformer.transformer.14.norm0.beta", "motion_transformer.transformer.15.fc0.fc.weight", "motion_transformer.transformer.15.fc0.fc.bias", "motion_transformer.transformer.15.norm0.alpha", "motion_transformer.transformer.15.norm0.beta", "motion_transformer.transformer.16.fc0.fc.weight", "motion_transformer.transformer.16.fc0.fc.bias", "motion_transformer.transformer.16.norm0.alpha", "motion_transformer.transformer.16.norm0.beta", "motion_transformer.transformer.17.fc0.fc.weight", "motion_transformer.transformer.17.fc0.fc.bias", "motion_transformer.transformer.17.norm0.alpha", "motion_transformer.transformer.17.norm0.beta", "motion_transformer.transformer.18.fc0.fc.weight", "motion_transformer.transformer.18.fc0.fc.bias", "motion_transformer.transformer.18.norm0.alpha", "motion_transformer.transformer.18.norm0.beta", "motion_transformer.transformer.19.fc0.fc.weight", "motion_transformer.transformer.19.fc0.fc.bias", "motion_transformer.transformer.19.norm0.alpha", "motion_transformer.transformer.19.norm0.beta", "motion_transformer.transformer.20.fc0.fc.weight", "motion_transformer.transformer.20.fc0.fc.bias", "motion_transformer.transformer.20.norm0.alpha", "motion_transformer.transformer.20.norm0.beta", "motion_transformer.transformer.21.fc0.fc.weight", "motion_transformer.transformer.21.fc0.fc.bias", "motion_transformer.transformer.21.norm0.alpha", "motion_transformer.transformer.21.norm0.beta", "motion_transformer.transformer.22.fc0.fc.weight", "motion_transformer.transformer.22.fc0.fc.bias", "motion_transformer.transformer.22.norm0.alpha", "motion_transformer.transformer.22.norm0.beta", "motion_transformer.transformer.23.fc0.fc.weight", "motion_transformer.transformer.23.fc0.fc.bias", "motion_transformer.transformer.23.norm0.alpha", "motion_transformer.transformer.23.norm0.beta", "motion_transformer.transformer.24.fc0.fc.weight", "motion_transformer.transformer.24.fc0.fc.bias", "motion_transformer.transformer.24.norm0.alpha", "motion_transformer.transformer.24.norm0.beta", "motion_transformer.transformer.25.fc0.fc.weight", "motion_transformer.transformer.25.fc0.fc.bias", "motion_transformer.transformer.25.norm0.alpha", "motion_transformer.transformer.25.norm0.beta", "motion_transformer.transformer.26.fc0.fc.weight", "motion_transformer.transformer.26.fc0.fc.bias", "motion_transformer.transformer.26.norm0.alpha", "motion_transformer.transformer.26.norm0.beta", "motion_transformer.transformer.27.fc0.fc.weight", "motion_transformer.transformer.27.fc0.fc.bias", "motion_transformer.transformer.27.norm0.alpha", "motion_transformer.transformer.27.norm0.beta", "motion_transformer.transformer.28.fc0.fc.weight", "motion_transformer.transformer.28.fc0.fc.bias", "motion_transformer.transformer.28.norm0.alpha", "motion_transformer.transformer.28.norm0.beta", "motion_transformer.transformer.29.fc0.fc.weight", "motion_transformer.transformer.29.fc0.fc.bias", "motion_transformer.transformer.29.norm0.alpha", "motion_transformer.transformer.29.norm0.beta", "motion_transformer.transformer.30.fc0.fc.weight", "motion_transformer.transformer.30.fc0.fc.bias", "motion_transformer.transformer.30.norm0.alpha", "motion_transformer.transformer.30.norm0.beta", "motion_transformer.transformer.31.fc0.fc.weight", "motion_transformer.transformer.31.fc0.fc.bias", "motion_transformer.transformer.31.norm0.alpha", "motion_transformer.transformer.31.norm0.beta", "motion_transformer.transformer.32.fc0.fc.weight", "motion_transformer.transformer.32.fc0.fc.bias", "motion_transformer.transformer.32.norm0.alpha", "motion_transformer.transformer.32.norm0.beta", "motion_transformer.transformer.33.fc0.fc.weight", "motion_transformer.transformer.33.fc0.fc.bias", "motion_transformer.transformer.33.norm0.alpha", "motion_transformer.transformer.33.norm0.beta", "motion_transformer.transformer.34.fc0.fc.weight", "motion_transformer.transformer.34.fc0.fc.bias", "motion_transformer.transformer.34.norm0.alpha", "motion_transformer.transformer.34.norm0.beta", "motion_transformer.transformer.35.fc0.fc.weight", "motion_transformer.transformer.35.fc0.fc.bias", "motion_transformer.transformer.35.norm0.alpha", "motion_transformer.transformer.35.norm0.beta", "motion_transformer.transformer.36.fc0.fc.weight", "motion_transformer.transformer.36.fc0.fc.bias", "motion_transformer.transformer.36.norm0.alpha", "motion_transformer.transformer.36.norm0.beta", "motion_transformer.transformer.37.fc0.fc.weight", "motion_transformer.transformer.37.fc0.fc.bias", "motion_transformer.transformer.37.norm0.alpha", "motion_transformer.transformer.37.norm0.beta", "motion_transformer.transformer.38.fc0.fc.weight", "motion_transformer.transformer.38.fc0.fc.bias", "motion_transformer.transformer.38.norm0.alpha", "motion_transformer.transformer.38.norm0.beta", "motion_transformer.transformer.39.fc0.fc.weight", "motion_transformer.transformer.39.fc0.fc.bias", "motion_transformer.transformer.39.norm0.alpha", "motion_transformer.transformer.39.norm0.beta", "motion_transformer.transformer.40.fc0.fc.weight", "motion_transformer.transformer.40.fc0.fc.bias", "motion_transformer.transformer.40.norm0.alpha", "motion_transformer.transformer.40.norm0.beta", "motion_transformer.transformer.41.fc0.fc.weight", "motion_transformer.transformer.41.fc0.fc.bias", "motion_transformer.transformer.41.norm0.alpha", "motion_transformer.transformer.41.norm0.beta", "motion_transformer.transformer.42.fc0.fc.weight", "motion_transformer.transformer.42.fc0.fc.bias", "motion_transformer.transformer.42.norm0.alpha", "motion_transformer.transformer.42.norm0.beta", "motion_transformer.transformer.43.fc0.fc.weight", "motion_transformer.transformer.43.fc0.fc.bias", "motion_transformer.transformer.43.norm0.alpha", "motion_transformer.transformer.43.norm0.beta", "motion_transformer.transformer.44.fc0.fc.weight", "motion_transformer.transformer.44.fc0.fc.bias", "motion_transformer.transformer.44.norm0.alpha", "motion_transformer.transformer.44.norm0.beta", "motion_transformer.transformer.45.fc0.fc.weight", "motion_transformer.transformer.45.fc0.fc.bias", "motion_transformer.transformer.45.norm0.alpha", "motion_transformer.transformer.45.norm0.beta", "motion_transformer.transformer.46.fc0.fc.weight", "motion_transformer.transformer.46.fc0.fc.bias", "motion_transformer.transformer.46.norm0.alpha", "motion_transformer.transformer.46.norm0.beta", "motion_transformer.transformer.47.fc0.fc.weight", "motion_transformer.transformer.47.fc0.fc.bias", "motion_transformer.transformer.47.norm0.alpha", "motion_transformer.transformer.47.norm0.beta".

Can you please tell me how can I solve that? thanks in advance.

GUO-W commented 1 year ago

Thanks for pointing this out. This is because we renamed the layer when cleaning the code for release. You could directly replace the "motion_transformer.transformer" with "motion_mlp.mlps" and it should be fine.

bhomaidan1990 commented 1 year ago

I didn't manage to find motion_transformer.transformer or mlps in the motion_mlp, can you please refer to that line in the code, thanks.

GUO-W commented 1 year ago

You could either replace "motion_mlp" with "motion_transformer" in the code; or after loading the model, change the keys of the loaded model "motion_transformer" to "motion_mlp".
I will modify the keys of the model and upload it later today.

bhomaidan1990 commented 1 year ago

You could either replace "motion_mlp" with "motion_transformer" in the code; or after loading the model, change the keys of the loaded model "motion_transformer" to "motion_mlp". I will modify the keys of the model and upload it later today.

If you please can also upload your pre-trained AMAAS model it will be kind of you, thanks in advance.

penway commented 1 year ago

I added a line to replace keys in state_dict and it worked.

state_dict = {k.replace("motion_transformer.transformer", "motion_mlp.mlps"): v for k, v in state_dict.items()}

Which is after line 108 in baseline_h36m/test.py

    model = Model(config)

    state_dict = torch.load(args.model_pth)

    # line to add
    state_dict = {k.replace("motion_transformer.transformer", "motion_mlp.mlps"): v for k, v in state_dict.items()}

    model.load_state_dict(state_dict, strict=True)
    model.eval()
    model.cuda()
Tong-Hao741 commented 6 months ago

but the reuslt is strang... [13.4, 24.0, 47.7, 58.4, 78.6, 92.1, 105.2, 112.0] is so low than paper result QAQ

GUO-W commented 6 months ago

Please see our latest update here, we upload our newly trained model using the exact same script of this repo. With the newly uploaded models, this issue is solved.