ThalesGroup / ConceptBERT

Implementation of ConceptBert: Concept-Aware Representation for Visual Question Answering
https://github.com/ThalesGroup/ConceptBERT
Other
25 stars 11 forks source link

pytorch size mismatch issue #15

Open Andrew82106 opened 1 year ago

Andrew82106 commented 1 year ago

We use the following command for fine-tuning the OKVQA model:

python3 -u train_tasks.py --model_version 3 --bert_model=bert-base-uncased \
--from_pretrained=/workspace/ConceptBERT-master/conceptBert/vilbert/data2/kilbert_base_model/pytorch_model_11.bin \
--from_pretrained_conceptBert=/workspace/ConceptBERT-master/conceptBert/outputs/train1_vqa_trained_model/VQA_bert_base_6layer_6conect/pytorch_model_18.bin \
--output_dir=/workspace/ConceptBERT-master/conceptBert/outputs/train2_okvqa_trained_model/ \
--summary_writer =/workspace/ConceptBERT-master/conceptBert/outputs/tensorboards/ \
--num_workers 0 \
--tasks 42

But we encountered the following problem:

Traceback (most recent call last):
  File "train_tasks.py", line 596, in <module>
    main()
  File "train_tasks.py", line 382, in main
    split="train",
  File "/workspace/ConceptBERT-master/conceptBert/conceptbert_models.py", line 64, in __init__
    pretrained_model_name_or_path=from_pretrained, config=config, num_labels=num_labels, split=split, default_gpu=default_gpu,
  File "/workspace/ConceptBERT-master/conceptBert/vilbert/vilbert.py", line 1508, in from_pretrained
    model.__class__.__name__, "\n\t".join(error_msgs)
RuntimeError: Error(s) in loading state_dict for VILBertForVLTasks:
        size mismatch for vil_prediction.main.3.bias: copying a param with shape torch.Size([3129]) from checkpoint, the shape in current model is torch.Size([5117]).
        size mismatch for vil_prediction.main.3.weight_v: copying a param with shape torch.Size([3129, 2048]) from checkpoint, the shape in current model is torch.Size([5117, 2048]).

I want to know if this problem is related to the file "/workspace/ConceptBERT-master/conceptBert/vilbert/data2/OKVQA/mscoco_train2014_annotations.json"? If yes, where can I download the correct file?

coreeey commented 1 year ago

Have you solved this problem yet? Is it due to the different number of answers?