Anwarvic
commented
3 months ago Hi @RichardLin1999,
Sorry that you are facing this issue! This issue is about a missmatch between the decoder's expected dimension (which is 1024 in this case) and the decoder's size of the loaded checkpoint (which is 765 in this case).
So, sre you trying to load a finetuned checkpoint from the provided checkpoints in our repo, or this is your fine-tuned checkpoint? If it's selected from our repo, please point out to which one you used. If it's your checkpoint, please upload it to any storage service (e.g. GoogleDrive) so we can replicate your error.
Traceback (most recent call last): File "/home/lpl/muavic/demo/run_demo.py", line 220, in
AV_RESOURCES = load_av_models(args.av_models_path)
File "/home/lpl/muavic/demo/demo_utils.py", line 65, in load_avmodels
models, , task = checkpoint_utils.load_model_ensemble_and_task(
File "/home/lpl/av_hubert/fairseq/fairseq/checkpoint_utils.py", line 447, in load_model_ensemble_and_task
model.load_state_dict(
File "/home/lpl/av_hubert/fairseq/fairseq/models/fairseq_model.py", line 125, in load_state_dict
return super().load_state_dict(new_state_dict, strict)
File "/usr/local/lib/python3.9/site-packages/torch/nn/modules/module.py", line 2041, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for AVHubertSeq2Seq:
size mismatch for decoder.layers.0.encoder_attn.k_proj.weight: copying a param with shape torch.Size([768, 1024]) from checkpoint, the shape in current model is torch.Size([768, 768]).
size mismatch for decoder.layers.0.encoder_attn.v_proj.weight: copying a param with shape torch.Size([768, 1024]) from checkpoint, the shape in current model is torch.Size([768, 768]).
size mismatch for decoder.layers.1.encoder_attn.k_proj.weight: copying a param with shape torch.Size([768, 1024]) from checkpoint, the shape in current model is torch.Size([768, 768]).
size mismatch for decoder.layers.1.encoder_attn.v_proj.weight: copying a param with shape torch.Size([768, 1024]) from checkpoint, the shape in current model is torch.Size([768, 768]).
size mismatch for decoder.layers.2.encoder_attn.k_proj.weight: copying a param with shape torch.Size([768, 1024]) from checkpoint, the shape in current model is torch.Size([768, 768]).
size mismatch for decoder.layers.2.encoder_attn.v_proj.weight: copying a param with shape torch.Size([768, 1024]) from checkpoint, the shape in current model is torch.Size([768, 768]).
size mismatch for decoder.layers.3.encoder_attn.k_proj.weight: copying a param with shape torch.Size([768, 1024]) from checkpoint, the shape in current model is torch.Size([768, 768]).
size mismatch for decoder.layers.3.encoder_attn.v_proj.weight: copying a param with shape torch.Size([768, 1024]) from checkpoint, the shape in current model is torch.Size([768, 768]).
size mismatch for decoder.layers.4.encoder_attn.k_proj.weight: copying a param with shape torch.Size([768, 1024]) from checkpoint, the shape in current model is torch.Size([768, 768]).
size mismatch for decoder.layers.4.encoder_attn.v_proj.weight: copying a param with shape torch.Size([768, 1024]) from checkpoint, the shape in current model is torch.Size([768, 768]).
size mismatch for decoder.layers.5.encoder_attn.k_proj.weight: copying a param with shape torch.Size([768, 1024]) from checkpoint, the shape in current model is torch.Size([768, 768]).
size mismatch for decoder.layers.5.encoder_attn.v_proj.weight: copying a param with shape torch.Size([768, 1024]) from checkpoint, the shape in current model is torch.Size([768, 768]).