Pretraining for I-CyCLIP and C-CyCLIP

[jonathan-roberts1]

Thanks for releasing your code and checkpoints!

The Google Drive checkpoints folder contains checkpoints for the I-CyCLIP and C-CyCLIP models, how many examples were these models trained on? My guess would be: CC3M data only ~ 2.6M datapoints, but I can't see an explicit mention in the repo/paper.

[Hritikbansal]

Hi @jonathan-roberts1,

You are right! They are pretrained on 2.6M data points from CC3M dataset.

[sarahESL]

Is CyCLIP trained from scratch or it starts from the CLIP embeddings?

[goel-shashank]

Hi @sarahESL, it is trained from scratch.