Closed bryanbocao closed 2 months ago
I assum there is a dot product step that represents two modalities' similarity like np.dot(I_embed, T_embed.T)
from the outputs outputs = model(**inputs)
, then it makes sense to me. Will check the code inside model()
when I have a chance later. Thanks!
Dear Authors,
Thanks for open-sourcing the work!
I tried to understand the
image-text similarity score
:in Quick Start:
Specifically, I have difficulty understanding using
logits
to represent a image-text similarity score. It would be great to further explain that.To me, a image-text similarity score is computed by the similarity between image and text in the feature space. Two examples would be to measure the distance
||image_features - text_features ||
or cosine similaritycos(image_features - text_features)
from the following code: