search

vladan-stojnic / ZLaP

Code for Label Propagation for Zero-shot Classification with Vision-Language Models (CVPR2024)
MIT License
32 stars 3 forks source link

Evaluating inductive inference #2

Closed sehyunkwon closed 1 month ago

sehyunkwon commented 1 month ago

Hello! Thank you for your excellent work.

I enjoyed reading your paper, and I am currently running some experiments with your code.

However, in inductive mode, it seems that the accuracy is being measured using test_targets. i.e, acc = accuracy(scores, test_targets). Is this correct?

Also, could you please check again if the output shape of do_inductive_lp is (test_features.shape[0], num_classes)?

vladan-stojnic commented 1 month ago

Hi,

Thanks for the interest in our work. In both transductive and inductive setups we evaluate on the test set, so using test_targets is correct. The difference is that in the transductive setup test images are unlabeled nodes of the graph, while in the inductive setup unlabeled nodes of the graph are images from the training set and we get the predictions for test images by representing them using neighbors from the graph (Eq. (11) from the paper).

I have checked and the output of do_inductive_lp is (test_features.shape[0], num_classes). The output of do_inductive_lp is scores array that was defined with the expected shape here.

sehyunkwon commented 1 month ago

Oh I misunderstood. So you were referring to the nodes that make up the graph Laplacian as 'unlabeled nodes.' Now it's clear to me. Thanks!