search

hereismari / tensorflow-maml

TensorFlow 2.0 implementation of MAML.
Apache License 2.0
83 stars 20 forks source link

when calculating test loss #4

Closed algebra-cadabra closed 4 years ago

algebra-cadabra commented 4 years ago

Hello thanx form the code it helped alot!

i have some questions for calculating test loss in MAML

when your code calculates test loss in train_maml() for specific sine function in inner loop

are you using same data to calculate both training and test loss? shouldn't you sample new test data from same sine function to calculate test error like in the paper

thanx

algebra-cadabra commented 4 years ago

I think you are optimizing for dropping training errors in meta learning task whereas in the paper it says you should optimize for dropping test errors in Algorithm 2 / line 2 in the paper.

Thanx

shufflebyte commented 4 years ago

you are right. you should sample new sine waves (not only 10 new samples of the same wave) for validation. If you take a look into the code of Finn et al. you can see, that they sample from new sines too.

hereismari commented 4 years ago

Yes, thanks for the catch! Feel free to send a PR!

On Sun, Jan 12, 2020, 10:37 Florian Soulier notifications@github.com wrote:

you are right. you should sample new sine waves (not only 10 new samples of the same wave) for validation. If you take a look into the code of Finn et al. you can see, that they sample from new sines too.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/mari-linhares/tensorflow-maml/issues/4?email_source=notifications&email_token=AB6HP3F4YZMU3F7WMTNV4H3Q5LXG3A5CNFSM4KESC5F2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIWWWDA#issuecomment-573401868, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB6HP3AMXNV7LGWCU7NZU2TQ5LXG3ANCNFSM4KESC5FQ .

algebra-cadabra commented 4 years ago

Thanx to both of you ~~!