eucl distance function in Tab. 1b

facebookresearch / ToMe

A method to increase the speed and lower the memory footprint of existing vision transformers.

Other

957 stars 68 forks source link

eucl distance function in Tab. 1b #19

Closed ChenMnZ closed 1 year ago

ChenMnZ commented 1 year ago

Hello, Thank you for your interesting work!

I want to know the implementation of eucl distance function in Tab. 1b. I try to use scores = torch.cdist(a,b) to calculate the eucl distance, but the top-1 accuracy on ViT-L/16 with r=8 is only 61.50 by my implementation.

Thanks!

dbolya commented 1 year ago

Hi, thanks for your interest!

A couple of things:

Make sure metrics isn't normalized beforehand. Normalization is there to compute cosine similarity efficiently.
Euclidian distance is a distance measure, not a similarity measure. So, you need to use -torch.cdist(a,b) instead.

Of the top of my head, I can't think of anything else.

ChenMnZ commented 1 year ago

Thanks! I have successfully reproduced.