search

huawei-noah / Pretrained-Language-Model

Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.
3.03k stars 627 forks source link

wukong模型图文测试相似度太低 #204

Closed douzi0248 closed 2 years ago

douzi0248 commented 2 years ago

使用图文对进行测试，模型是vit_b模型进行测试，图文相似度[0.0937,0.0846,0.0857]

mengxj08 commented 2 years ago

您好，这个结果是logits，未做softmax吧，如果想根据logits计算图文相似度，建议follow CLIP工作的方式，* temperature parameter以后再softmax：

logits = (100 * image_features @ text_features.T).softmax(dim=-1)

quinwu commented 1 month ago

@douzi0248 最后问题解决了吗？我也遇到了同样的问题