search

UMass-Foundation-Model / 3D-LLM

Code for 3D-LLM: Injecting the 3D World into Large Language Models
MIT License
952 stars 59 forks source link

Runtime error on inference.py #76

Open jmstadt opened 4 months ago

jmstadt commented 4 months ago

Hello, thank you for your excellent work.

I am trying to run inference.py but when I run line 34:

model.load_state_dict(checkpoint["model"], strict=False)

I get a runtime error:

RuntimeError: Error(s) in loading state_dict for Blip2T5:
    size mismatch for t5_model.shared.weight: copying a param with shape torch.Size([64868, 2048]) from checkpoint, the shape in current model is torch.Size([32128, 2048]).
    size mismatch for t5_model.encoder.embed_tokens.weight: copying a param with shape torch.Size([64868, 2048]) from checkpoint, the shape in current model is torch.Size([32128, 2048]).
    size mismatch for t5_model.decoder.embed_tokens.weight: copying a param with shape torch.Size([64868, 2048]) from checkpoint, the shape in current model is torch.Size([32128, 2048]).
    size mismatch for t5_model.lm_head.weight: copying a param with shape torch.Size([64868, 2048]) from checkpoint, the shape in current model is torch.Size([32128, 2048]).

Appreciate any thoughts or guidance. I loaded both the v2 checkpoint and the v2.1 checkpoint with the same result.

jmstadt commented 4 months ago

NVM, per stackoverflow: model = torch.nn.DataParallel(model) https://stackoverflow.com/questions/61909973/pytorch-load-incompatiblekeys Running on CPU, that got me through that line of code.