search

salesforce / ALPRO

Align and Prompt: Video-and-Language Pre-training with Entity Prompts
BSD 3-Clause "New" or "Revised" License
185 stars 18 forks source link

How to initialize the model with ALBEF when finetuning? #5

Closed cdqncn closed 2 years ago

cdqncn commented 2 years ago

I noticed that your codes have a comment "# NOTE strict is False if loaded from ALBEF ckpt", when I did finetune on downstream tasks, I want to try to use the released ALBEF.pth to initialize the model, but some error appeared. Could you help me? Thanks!

dxli94 commented 2 years ago

The comment is deprecated.

ALPRO has a different model architecture from ALBEF. Reusing some weights from ALBEF for initialization is possible but currently not supported by the codebase.

If desired, you may manually load weights from ALBEF model state_dict().

Thanks.