salesforce / ALBEF

Code for ALBEF: a new vision-language pre-training method
BSD 3-Clause "New" or "Revised" License
1.45k stars 193 forks source link

pretrain task #122

Open allenzg opened 1 year ago

allenzg commented 1 year ago

Your pre-trained model introduced from x.bert to BertforMasked LM does not have this class, I tried to introduce from the transformer, I can't withdraw the first six layers are text encoders, the last six layers are fusion encoders, how to withdraw them. Error additional parameter 'mode'