Evaluation

[xialeiliu]

It's a great work on comparison among softmax, siameses and triplets networks, I am wondering what kind of metric do you think is possible to compare different methods fairly in stead of only qualitative results. I saw it's on TODO list, do you have any idea about it?

[adambielski]

Sure, I think a good thing to do here would be to evaluate it on a one-shot learning task, e.g. Omniglot. Train the embeddings on a subset on classes, than extract embeddings of the other classes treating them as examplar images and query images. Then check accuracies e.g. for kNN. There are better approaches though for this kind of problems now (e.g. prototypical networks)

[xialeiliu]

Thanks.