search

learnables / learn2learn

A PyTorch Library for Meta-learning Research
http://learn2learn.net
MIT License
2.61k stars 350 forks source link

examples/text/news_topic_classification.py - has it been tested? #379

Closed jkang1640 closed 1 year ago

jkang1640 commented 1 year ago

Hello,

I tried the script but the loss / accuracy does not get improve till the end of training and it seems that the model didn't learn anything..

Looking closely into the code, it seems that validation set is wrongly used as a query set... (train dataset is not divided into a support and query set. Instead, validation set is used as a query set and loss on the validation set is used to update the model)

Has this code been tested already before release? If so, did you observe that the the model accuracy improves as the training goes?

Thank you.

seba-1511 commented 1 year ago

Hello @jkang1640,

Indeed, we were never able to get MAML to work well on text. See #96 for details.

This makes me think I should remove the text example, since it seems to be below the standard of the library. I'll leave the issue open until it's removed. If you want to submit a PR to fix the script (or remove it) I'll happily merge it.

seba-1511 commented 1 year ago

Update: I've decided to remove it in #400 since there isn't much interest. Closing this issue now.