Missing Keys in State Dict

[nmalboubi]

Receiving this Error when running anchor captioner on test data - would appreciate any assistance:

2021-07-11T09:31:47 ERROR: Error(s) in loading state_dict for M4CCaptioner: Missing key(s) in state_dict: "graph_proposal_net.t2t.encoder.layer.0.attention.self.query.weight", "graph_proposal_net.t2t.encoder.layer.0.attention.self.query.bias", "graph_proposal_net.t2t.encoder.layer.0.attention.self.key.weight", "graph_proposal_net.t2t.encoder.layer.0.attention.self.key.bias", "graph_proposal_net.t2t.encoder.layer.0.attention.self.value.weight", "graph_proposal_net.t2t.encoder.layer.0.attention.self.value.bias", "graph_proposal_net.t2t.encoder.layer.0.attention.output.dense.weight", "graph_proposal_net.t2t.encoder.layer.0.attention.output.dense.bias", "graph_proposal_net.t2t.encoder.layer.0.attention.output.LayerNorm.weight", "graph_proposal_net.t2t.encoder.layer.0.attention.output.LayerNorm.bias", "graph_proposal_net.t2t.encoder.layer.0.intermediate.dense.weight", "graph_proposal_net.t2t.encoder.layer.0.intermediate.dense.bias", "graph_proposal_net.t2t.encoder.layer.0.output.dense.weight", "graph_proposal_net.t2t.encoder.layer.0.output.dense.bias", "graph_proposal_net.t2t.encoder.layer.0.output.LayerNorm.weight", "graph_proposal_net.t2t.encoder.layer.0.output.LayerNorm.bias", "graph_proposal_net.t2t.encoder.layer.1.attention.self.query.weight", "graph_proposal_net.t2t.encoder.layer.1.attention.self.query.bias", "graph_proposal_net.t2t.encoder.layer.1.attention.self.key.weight", "graph_proposal_net.t2t.encoder.layer.1.attention.self.key.bias", "graph_proposal_net.t2t.encoder.layer.1.attention.self.value.weight", "graph_proposal_net.t2t.encoder.layer.1.attention.self.value.bias", "graph_proposal_net.t2t.encoder.layer.1.attention.output.dense.weight", "graph_proposal_net.t2t.encoder.layer.1.attention.output.dense.bias", "graph_proposal_net.t2t.encoder.layer.1.attention.output.LayerNorm.weight", "graph_proposal_net.t2t.encoder.layer.1.attention.output.LayerNorm.bias", "graph_proposal_net.t2t.encoder.layer.1.intermediate.dense.weight", "graph_proposal_net.t2t.encoder.layer.1.intermediate.dense.bias", "graph_proposal_net.t2t.encoder.layer.1.output.dense.weight", "graph_proposal_net.t2t.encoder.layer.1.output.dense.bias", "graph_proposal_net.t2t.encoder.layer.1.output.LayerNorm.weight", "graph_proposal_net.t2t.encoder.layer.1.output.LayerNorm.bias", "graph_proposal_net.t2t.encoder.layer.2.attention.self.query.weight", "graph_proposal_net.t2t.encoder.layer.2.attention.self.query.bias", "graph_proposal_net.t2t.encoder.layer.2.attention.self.key.weight", "graph_proposal_net.t2t.encoder.layer.2.attention.self.key.bias", "graph_proposal_net.t2t.encoder.layer.2.attention.self.value.weight", "graph_proposal_net.t2t.encoder.layer.2.attention.self.value.bias", "graph_proposal_net.t2t.encoder.layer.2.attention.output.dense.weight", "graph_proposal_net.t2t.encoder.layer.2.attention.output.dense.bias", "graph_proposal_net.t2t.encoder.layer.2.attention.output.LayerNorm.weight", "graph_proposal_net.t2t.encoder.layer.2.attention.output.LayerNorm.bias", "graph_proposal_net.t2t.encoder.layer.2.intermediate.dense.weight", "graph_proposal_net.t2t.encoder.layer.2.intermediate.dense.bias", "graph_proposal_net.t2t.encoder.layer.2.output.dense.weight", "graph_proposal_net.t2t.encoder.layer.2.output.dense.bias", "graph_proposal_net.t2t.encoder.layer.2.output.LayerNorm.weight", "graph_proposal_net.t2t.encoder.layer.2.output.LayerNorm.bias", "graph_proposal_net.t2t.encoder.layer.3.attention.self.query.weight", "graph_proposal_net.t2t.encoder.layer.3.attention.self.query.bias", "graph_proposal_net.t2t.encoder.layer.3.attention.self.key.weight", "graph_proposal_net.t2t.encoder.layer.3.attention.self.key.bias", "graph_proposal_net.t2t.encoder.layer.3.attention.self.value.weight", "graph_proposal_net.t2t.encoder.layer.3.attention.self.value.bias", "graph_proposal_net.t2t.encoder.layer.3.attention.output.dense.weight", "graph_proposal_net.t2t.encoder.layer.3.attention.output.dense.bias", "graph_proposal_net.t2t.encoder.layer.3.attention.output.LayerNorm.weight", "graph_proposal_net.t2t.encoder.layer.3.attention.output.LayerNorm.bias", "graph_proposal_net.t2t.encoder.layer.3.intermediate.dense.weight", "graph_proposal_net.t2t.encoder.layer.3.intermediate.dense.bias", "graph_proposal_net.t2t.encoder.layer.3.output.dense.weight", "graph_proposal_net.t2t.encoder.layer.3.output.dense.bias", "graph_proposal_net.t2t.encoder.layer.3.output.LayerNorm.weight", "graph_proposal_net.t2t.encoder.layer.3.output.LayerNorm.bias", "graph_proposal_net.anchor_fc.0.weight", "graph_proposal_net.anchor_fc.0.bias", "graph_proposal_net.anchor_fc.2.weight", "graph_proposal_net.anchor_fc.2.bias", "graph_proposal_net.graph_fc.0.weight", "graph_proposal_net.graph_fc.0.bias", "graph_proposal_net.graph_fc.2.weight", "graph_proposal_net.graph_fc.2.bias", "graph_proposal_net.rnn.weight_ih_l0", "graph_proposal_net.rnn.weight_hh_l0", "graph_proposal_net.rnn.bias_ih_l0", "graph_proposal_net.rnn.bias_hh_l0", "r_mmt.prev_pred_embeddings.position_embeddings.weight", "r_mmt.prev_pred_embeddings.ans_layer_norm.weight", "r_mmt.prev_pred_embeddings.ans_layer_norm.bias", "r_mmt.prev_pred_embeddings.emb_layer_norm.weight", "r_mmt.prev_pred_embeddings.emb_layer_norm.bias", "r_mmt.encoder.layer.0.attention.self.query.weight", "r_mmt.encoder.layer.0.attention.self.query.bias", "r_mmt.encoder.layer.0.attention.self.key.weight", "r_mmt.encoder.layer.0.attention.self.key.bias", "r_mmt.encoder.layer.0.attention.self.value.weight", "r_mmt.encoder.layer.0.attention.self.value.bias", "r_mmt.encoder.layer.0.attention.output.dense.weight", "r_mmt.encoder.layer.0.attention.output.dense.bias", "r_mmt.encoder.layer.0.attention.output.LayerNorm.weight", "r_mmt.encoder.layer.0.attention.output.LayerNorm.bias", "r_mmt.encoder.layer.0.intermediate.dense.weight", "r_mmt.encoder.layer.0.intermediate.dense.bias", "r_mmt.encoder.layer.0.output.dense.weight", "r_mmt.encoder.layer.0.output.dense.bias", "r_mmt.encoder.layer.0.output.LayerNorm.weight", "r_mmt.encoder.layer.0.output.LayerNorm.bias", "r_mmt.encoder.layer.1.attention.self.query.weight", "r_mmt.encoder.layer.1.attention.self.query.bias", "r_mmt.encoder.layer.1.attention.self.key.weight", "r_mmt.encoder.layer.1.attention.self.key.bias", "r_mmt.encoder.layer.1.attention.self.value.weight", "r_mmt.encoder.layer.1.attention.self.value.bias", "r_mmt.encoder.layer.1.attention.output.dense.weight", "r_mmt.encoder.layer.1.attention.output.dense.bias", "r_mmt.encoder.layer.1.attention.output.LayerNorm.weight", "r_mmt.encoder.layer.1.attention.output.LayerNorm.bias", "r_mmt.encoder.layer.1.intermediate.dense.weight", "r_mmt.encoder.layer.1.intermediate.dense.bias", "r_mmt.encoder.layer.1.output.dense.weight", "r_mmt.encoder.layer.1.output.dense.bias", "r_mmt.encoder.layer.1.output.LayerNorm.weight", "r_mmt.encoder.layer.1.output.LayerNorm.bias", "r_mmt.encoder.layer.2.attention.self.query.weight", "r_mmt.encoder.layer.2.attention.self.query.bias", "r_mmt.encoder.layer.2.attention.self.key.weight", "r_mmt.encoder.layer.2.attention.self.key.bias", "r_mmt.encoder.layer.2.attention.self.value.weight", "r_mmt.encoder.layer.2.attention.self.value.bias", "r_mmt.encoder.layer.2.attention.output.dense.weight", "r_mmt.encoder.layer.2.attention.output.dense.bias", "r_mmt.encoder.layer.2.attention.output.LayerNorm.weight", "r_mmt.encoder.layer.2.attention.output.LayerNorm.bias", "r_mmt.encoder.layer.2.intermediate.dense.weight", "r_mmt.encoder.layer.2.intermediate.dense.bias", "r_mmt.encoder.layer.2.output.dense.weight", "r_mmt.encoder.layer.2.output.dense.bias", "r_mmt.encoder.layer.2.output.LayerNorm.weight", "r_mmt.encoder.layer.2.output.LayerNorm.bias", "r_mmt.encoder.layer.3.attention.self.query.weight", "r_mmt.encoder.layer.3.attention.self.query.bias", "r_mmt.encoder.layer.3.attention.self.key.weight", "r_mmt.encoder.layer.3.attention.self.key.bias", "r_mmt.encoder.layer.3.attention.self.value.weight", "r_mmt.encoder.layer.3.attention.self.value.bias", "r_mmt.encoder.layer.3.attention.output.dense.weight", "r_mmt.encoder.layer.3.attention.output.dense.bias", "r_mmt.encoder.layer.3.attention.output.LayerNorm.weight", "r_mmt.encoder.layer.3.attention.output.LayerNorm.bias", "r_mmt.encoder.layer.3.intermediate.dense.weight", "r_mmt.encoder.layer.3.intermediate.dense.bias", "r_mmt.encoder.layer.3.output.dense.weight", "r_mmt.encoder.layer.3.output.dense.bias", "r_mmt.encoder.layer.3.output.LayerNorm.weight", "r_mmt.encoder.layer.3.output.LayerNorm.bias". Unexpected key(s) in state_dict: "text_bert.embeddings.word_embeddings.weight", "text_bert.embeddings.position_embeddings.weight", "text_bert.embeddings.token_type_embeddings.weight", "text_bert.embeddings.LayerNorm.weight", "text_bert.embeddings.LayerNorm.bias", "text_bert.encoder.layer.0.attention.self.query.weight", "text_bert.encoder.layer.0.attention.self.query.bias", "text_bert.encoder.layer.0.attention.self.key.weight", "text_bert.encoder.layer.0.attention.self.key.bias", "text_bert.encoder.layer.0.attention.self.value.weight", "text_bert.encoder.layer.0.attention.self.value.bias", "text_bert.encoder.layer.0.attention.output.dense.weight", "text_bert.encoder.layer.0.attention.output.dense.bias", "text_bert.encoder.layer.0.attention.output.LayerNorm.weight", "text_bert.encoder.layer.0.attention.output.LayerNorm.bias", "text_bert.encoder.layer.0.intermediate.dense.weight", "text_bert.encoder.layer.0.intermediate.dense.bias", "text_bert.encoder.layer.0.output.dense.weight", "text_bert.encoder.layer.0.output.dense.bias", "text_bert.encoder.layer.0.output.LayerNorm.weight", "text_bert.encoder.layer.0.output.LayerNorm.bias", "text_bert.encoder.layer.1.attention.self.query.weight", "text_bert.encoder.layer.1.attention.self.query.bias", "text_bert.encoder.layer.1.attention.self.key.weight", "text_bert.encoder.layer.1.attention.self.key.bias", "text_bert.encoder.layer.1.attention.self.value.weight", "text_bert.encoder.layer.1.attention.self.value.bias", "text_bert.encoder.layer.1.attention.output.dense.weight", "text_bert.encoder.layer.1.attention.output.dense.bias", "text_bert.encoder.layer.1.attention.output.LayerNorm.weight", "text_bert.encoder.layer.1.attention.output.LayerNorm.bias", "text_bert.encoder.layer.1.intermediate.dense.weight", "text_bert.encoder.layer.1.intermediate.dense.bias", "text_bert.encoder.layer.1.output.dense.weight", "text_bert.encoder.layer.1.output.dense.bias", "text_bert.encoder.layer.1.output.LayerNorm.weight", "text_bert.encoder.layer.1.output.LayerNorm.bias", "text_bert.encoder.layer.2.attention.self.query.weight", "text_bert.encoder.layer.2.attention.self.query.bias", "text_bert.encoder.layer.2.attention.self.key.weight", "text_bert.encoder.layer.2.attention.self.key.bias", "text_bert.encoder.layer.2.attention.self.value.weight", "text_bert.encoder.layer.2.attention.self.value.bias", "text_bert.encoder.layer.2.attention.output.dense.weight", "text_bert.encoder.layer.2.attention.output.dense.bias", "text_bert.encoder.layer.2.attention.output.LayerNorm.weight", "text_bert.encoder.layer.2.attention.output.LayerNorm.bias", "text_bert.encoder.layer.2.intermediate.dense.weight", "text_bert.encoder.layer.2.intermediate.dense.bias", "text_bert.encoder.layer.2.output.dense.weight", "text_bert.encoder.layer.2.output.dense.bias", "text_bert.encoder.layer.2.output.LayerNorm.weight", "text_bert.encoder.layer.2.output.LayerNorm.bias". Traceback (most recent call last): File "tools/run.py", line 102, in run() File "tools/run.py", line 90, in run trainer.load() File "/home/ec2-user/AnchorCaptioner/pythia/trainers/base_trainer.py", line 65, in load self.load_extras() File "/home/ec2-user/AnchorCaptioner/pythia/trainers/base_trainer.py", line 187, in load_extras self.checkpoint.load_state_dict() File "/home/ec2-user/AnchorCaptioner/pythia/utils/checkpoint.py", line 63, in load_state_dict self._load(tp.resume_file) File "/home/ec2-user/AnchorCaptioner/pythia/utils/checkpoint.py", line 117, in _load self.trainer.model.load_state_dict(final_dict) File "/home/ec2-user/anaconda3/lib/python3.7/site-packages/torch-1.9.0-py3.7-linux-x86_64.egg/torch/nn/modules/module.py", line 1407, in load_state_dict self.class.name, "\n\t".join(error_msgs)))

[guanghuixu]

I guess you load the pretrained weights of M4C to our model. You need to train the model use our code, instead of load the M4C

[nmalboubi]

Oh got it - so I can't run the pre-trained model on the test data set just to see performance? And the pre-trained model is just the raw M4C model, not a pre-trained version of the Anchor Captioner - is that correct?

[guanghuixu]

yes