Currently llama_proj_model is not passed to __init__ in from_config of VideoLLAMA
As a result the checkpoint is never loaded here. Even after this is fixed, model is also not defined here.
From the keys (llama_proj.weight) of the weights seems that the correct code is self.load_state_dict(llama_proj_weight['model'], strict=False)
Hi,
Currently
llama_proj_model
is not passed to__init__
infrom_config
ofVideoLLAMA
As a result the checkpoint is never loaded here. Even after this is fixed,model
is also not defined here. From the keys (llama_proj.weight
) of the weights seems that the correct code isself.load_state_dict(llama_proj_weight['model'], strict=False)