BigML-CS-UCLA / RoCLIP

Robust Contrastive Language-Image Pretraining against Data Poisoning and Backdoor Attacks
11 stars 2 forks source link

CLIP implementation different from openai's official repository? #4

Open Ashes-of-Midgard opened 8 months ago

Ashes-of-Midgard commented 8 months ago

I wonder why the code under ./model/pkgs/clip are slightly different from openai's official repository? For example, this is the architecture of Bottlenet in ./model/pkgs/clip/model.py image

However, this is the architecture of Bottlenet in "https://github.com/openai/CLIP/blob/main/clip/model.py" image

So what is the consideration behind these alteration? Why not use the implemented model by simply importing clip package? And are there any references for these changes? Also I'm confused about the 'pretrained' option. Since the model used is different from openai's implementation, why it is possible for the model to load the state dict value downloaded from openai's official URL? I tried to set it True and found the code worked, but I'm more confused than it didn't. Thanks for any help.