search

DAMO-NLP-SG / Video-LLaMA

[EMNLP 2023 Demo] Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding
BSD 3-Clause "New" or "Revised" License
2.77k stars 255 forks source link

llama_proj_model not passed to __init__ #92

Closed daahuang closed 1 year ago

daahuang commented 1 year ago

Hi,

Currently llama_proj_model is not passed to __init__ in from_config of VideoLLAMA As a result the checkpoint is never loaded here. Even after this is fixed, model is also not defined here. From the keys (llama_proj.weight) of the weights seems that the correct code is self.load_state_dict(llama_proj_weight['model'], strict=False)

hangzhang-nlp commented 1 year ago

Thanks a lot! It has been updated!