search

tsb0601 / MMVP

248 stars 7 forks source link

CLIP-blind pairs #10

Open mahtabbigverdi opened 4 months ago

mahtabbigverdi commented 4 months ago

Hi, Great work, have you also released the code for finding clip blind pairs?

clemneo commented 3 weeks ago

+1, or even if the code isn't released it would be good if you could share some implementation details!

Are you taking the cosine similarity of the activations at the [CLS] token position at the last layer?

tsb0601 commented 2 weeks ago

Hi, Sorry for the late reply. Of couse! We use the cosine similarity of the CLIPImage Encoder output: e.g. https://github.com/mlfoundations/open_clip/blob/58e4e39aaabc6040839b0d2a7e8bf20979e4558a/src/open_clip/model.py#L287