Test phase

[MarioProjects]

I am trying to use this (and Prototypical nets) on a custom problem and I dont understand the test time phase. I have 5000 classes with most of them with 1 picture only... So I train the model using 5 shot and 5 query for example (getting 4 pictures with data augmentation If I have not so many pictures) and then in test I dont understand how to do to map to the 5000 classes. Should I use query of all training size and query 1 by 1?

[dragen1860]

Hi, current few-shot learning is quite weak, hence we need to constrait the total categories number to small number, e.g. 5-way, 20-way. In your case, you need to sample N-way categories from your whole 5000 catgories. Since most of them owns only 1 picture, it's very challenging to do few-shot learning. You can still do N-way 1-shot exp. still. The data argumentation will create several samples for each category but it wont help too much. Big data is the key.

[MarioProjects]

Thanks Jackie! I thought that after send you the email. I thought that... I have a siamese getting great results, I could try to boost the results getting from the siamese for example the first 100 or 50 classes with hightest similarity and then use the relation net or prototypical network for example to classify the samples via 100-50 way... What you think? Thanks!

On Jan 2 2019, at 8:27 am, Jackie Loong notifications@github.com wrote:

Hi, current few-shot learning is quite weak, hence we need to constrait the total categories number to small number, e.g. 5-way, 20-way. In your case, you need to sample N-way categories from your whole 5000 catgories. Since most of them owns only 1 picture, it's very challenging to do few-shot learning. You can still do N-way 1-shot exp. still. The data argumentation will create several samples for each category but it wont help too much. Big data is the key.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub (https://link.getmailspring.com/link/1546416625.local-7298f513-f43b-v1.5.4-1d923cf4@getmailspring.com/0?redirect=https%3A%2F%2Fgithub.com%2Fdragen1860%2FLearningToCompare-Pytorch%2Fissues%2F11%23issuecomment-450799995&recipient=cmVwbHkrMDE2NGQ0ZjFjNmU4NTZiNDZkNjg4MTlhMzcxOTQ0MWRjMTMxYzc5MjA4ZmIzNzE3OTJjZjAwMDAwMDAxMTg0NDI5ZDM5MmExNjljZTE3OGM0YmFjQHJlcGx5LmdpdGh1Yi5jb20%3D), or mute the thread (https://link.getmailspring.com/link/1546416625.local-7298f513-f43b-v1.5.4-1d923cf4@getmailspring.com/1?redirect=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAWTU8YiL9FA8idq5Yz8NJY6rivCmj8K1ks5u_F9TgaJpZM4Zl1Zn&recipient=cmVwbHkrMDE2NGQ0ZjFjNmU4NTZiNDZkNjg4MTlhMzcxOTQ0MWRjMTMxYzc5MjA4ZmIzNzE3OTJjZjAwMDAwMDAxMTg0NDI5ZDM5MmExNjljZTE3OGM0YmFjQHJlcGx5LmdpdGh1Yi5jb20%3D).

[dragen1860]

Yes, it should work. But if you do it for research, you need follow these exp. setting in latest papers so that peers could review your algorithm fairly. If you are doing it for your individual projects, emmm, you also need more than 1 pictures per category to test its perrformance. If you use data argumentation to obtain several images, say, you get 2 images per category and then use one of them to train and the other to test, it is prone to achieve "good" performance. The truth is that, since the two images come from one single image by data argumentation, their distribuution is quite close and has low variance, it is relatively easy to classify for your trained classifier. So the "good" performance wont tell the truth. Make sure to use real images to test it.

[MarioProjects]

Hey Jackie thanks! Its for a university competition :) I have a training set and a test set where test are only individual pictures (I dont know if are 1, 2 or more by id) so I have to use test query 1 ( maybe 2 or 3 produced vía data augmentation to try if it helps). Thanks for your help. Maybe as siamese booster Will improve a little bit :D

On Wed, Jan 2, 2019 at 2:14 PM +0100, "Jackie Loong" notifications@github.com wrote:

Yes, it should work.

But if you do it for research, you need follow these exp. setting in latest papers so that peers could review your algorithm fairly.

If you are doing it for your individual projects, emmm, you also need more than 1 pictures per category to test its perrformance. If you use data argumentation to obtain several images, say, you get 2 images per category and then use one of them to train and the other to test, it is prone to achieve "good" performance. The truth is that, since the two images come from one single image by data argumentation, their distribuution is quite close and has low variance, it is relatively easy to classify for your trained classifier. So the "good" performance wont tell the truth.

Make sure to use real images to test it.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.