xmed-lab / CLIP_Surgery

CLIP Surgery for Better Explainability with Enhancement in Open-Vocabulary Tasks
368 stars 26 forks source link

How can I do zero-shot imagnet1k test with the surgery model? #16

Closed JianbangZ closed 1 year ago

Eli-YiLi commented 1 year ago

This issue has been discussed in #13 .

The answer is that "Single label like imagenet is not influenced, because architecture surgery is designed for explainability task, and feature surgery compute the redundant feature as a common bias for each class. Thus, giving a same bias doesn’t change the rank and accuracy, instead it influences scores across images and benefits mAP for multi-label".

If you want to test classification mAP, the first step is to record cosine similarity from the original path with feature surgery, and then eval with gt using package like torchmetrics.