search

dichotomies / proxy-nca

PyTorch Implementation of `No Fuss Distance Metric Learning using Proxies`.
MIT License
181 stars 33 forks source link

Other datasets and a remark #6

Closed artkorenev closed 6 years ago

artkorenev commented 6 years ago

Hi there,

You posted a comment on https://github.com/bnulihaixia/Deep_metric/issues/5#issuecomment-410479866 and I checked out your work here, nice job!

During my research around this loss, I found that CUB200 is the least challenging dataset to fit and I was able to reproduce the results on it quite well. However, I really struggled with the other datasets mentioned in the paper. Have you tried to use CARS196 dataset? It has relatively the same amount of training examples, however, achieving the same Recall@K metrics was more challenging.

One final remark is that you mention using InceptionV2 model however the authors stated that they used Inception-BN model which, as to my knowledge, are not the same thing. Although, as per the paper of InceptionV2 the 7x7 convolution was factorized into smaller ones and the presence of such convolution in your code reveals that it is Inception-BN rather than InceptionV2, which is correct (however I didn't check the whole code for the network thoroughly). It seems like you just mislabeled it.

dichotomies commented 6 years ago

If we compare R@1 and NMI for both datasets after training, it seems like that it's harder to train a network with CUB than Cars196 no? Or you mean in the particular case of reproducing the results with Proxy NCA?

I was able to reproduce the results for Cars, but with some adjustments.

Oh, so you implemented Proxy NCA as well? Have you uploaded it anywhere?

--

I think we agree that https://arxiv.org/pdf/1502.03167.pdf is about Inception V2. The authors conclude that BN plays an important role in reducing the internal covariate shift. This results in

the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin.

Furthermore, on p.8 they label the network BN-Inception. Therefore, I conclude that Inception V2 is BN-Inception.

Anyway, maybe I'm wrong for some reason. If both are not the same, could you point me to the paper of BN-Inception and Inception V2, respectively?

artkorenev commented 6 years ago

The network is trained and evaluated separately for each dataset and I was just saying that reproducing the results for Cars196 was significantly more challenging than for CUB200 for some reason. If you have the code reproducing the results on Cars196 (or Stanford Online Products) I would be happy to check it out.

ProxyNCA is quite straightforward to implement but I haven't yet uploaded my experiments anywhere. Maybe I will do it in a couple of months but it is not in a short list, sorry.

As for InceptionV2, it was introduced in this paper which is not the paper that introduced the Batch Normalization. By the way, I found a good explanation of discrepancies between different versions for Inception architectures here.

dichotomies commented 6 years ago

You are right, in Rethinking the Inception Architecture for Computer Vision they introduce Inception V2 and Inception V3. The major change for Inception V2 seems to be the factorization which you mentioned.

In the official TensorFlow repository they linked Inception V2 to the BN-Inception paper, so I thought that both must be the same.

Thanks for the hint!