search

mlfoundations / open_clip

An open source implementation of CLIP.
Other
9.93k stars 959 forks source link

How to initialize the encoder to pretrain CLIP？ #859

Closed shihuai closed 4 months ago

shihuai commented 5 months ago

HI, Thank you for your open source work. I am also very interested in your work. Regarding your work, I have a question: When you use Covnext as the image encoder to pre-train CLIP, the weights of convnext use ImageNet pre-trained initialization or random initialization?

rwightman commented 4 months ago

@shihuai imagenet pretrained weights were not used for training the convnexts here, this field is set to false https://github.com/mlfoundations/open_clip/blob/main/src/open_clip/model_configs/convnext_base.json#L5