search

tristandeleu / ntm-one-shot

One-shot Learning with Memory-Augmented Neural Networks
MIT License
421 stars 94 forks source link

Mann test results #10

Closed brat-eek closed 7 years ago

brat-eek commented 7 years ago

Does the implementation match the results as on the paper at test time ? Atleast are the accuracies even close because I dont seem to be even getting close. Thanks in adv.

tristandeleu commented 7 years ago

I did manage to get similar results for the 5 classes/episode experiment. Here are the results I got a few months back image

brat-eek commented 7 years ago

Thanks for the assurance! I might have screwed up while playing with it. :+1:

hspark84 commented 7 years ago

Why is the 1st instance high in the early episodes ? In the paper, that was low and flat in the early episodes.

tristandeleu commented 7 years ago

I can't explain the behavior of the 1st instance unfortunately. When I saw that first I thought I did something wrong with the way I'm computing the metrics, but as it gets closer to the results of the paper with a larger number of episodes, I think it's just a matter of (un)lucky initialization.

HyoungWooPark commented 7 years ago

Is the above results from training MANN with 5classes/episode experiment? I would get the same results from omniglot dataset by an instruction which you wrote. In your implementation, I cannot find how I can get the test results from trained MANN. How can I test it after training?