PCK Evaluation

[kampta]

I am unable to replicate results in Table 4 (Correspondence Evaluation on Spair71k). Since in the case of PCK evaluation, keypoints are provided for the source image, I find the closest point (according to the binned descriptor) in the second image within the "salient region". The numbers I get are close to zeros so there might be a mistake in my code. Are there any additional heuristics that you apply for this one-way correspondence?

[ShirAmir]

Hi @kampta

In order to compute PCK for a single image, we did the following for each given key point: (1) found the spatial patch it corresponds to when the image is processed by a ViT. (2) computed the binned descriptor from the 9th layer associated with that patch. (3) computed the cosine similarity between this descriptor and all the binned descriptors from the 9th layer in the target image. (4) we consider the spatial patch in the target image, that corresponds to the descriptor with highest cosine similarity. (5) we report the middle pixel of this patch as the resulting corresponding key point, and compare it to the GT in the target image.

You may find the recently added inspect_similarity.py helpful to find the correspondences. (conducts stages 1-5 for a single key point in a single image)

[kampta]

Thank you for your kind response. I discovered the bug in my script and now I am able to get PCK up to 15.5%. Here is the code that I used for PCK computation. If you get a chance, feel free to take a look at my code or share the code you used to compute PCK.

https://github.com/kampta/dino-vit-features/blob/spair/PCK_spair.ipynb

[kampta]

I discovered a bug in my script (I was using source keypoints instead of target keypoints for eval) and the results are comparable now. Thanks!

[yousafe007]

@ShirAmir Although it has been quite some while since evaluation was discussed, but could I ask you to share the evaluation code you used for PCK using the SPair dataset?