search

hoyeoplee / MeLU

178 stars 52 forks source link

Some questions about the evaluation process #6

Open yinxiaojian opened 5 years ago

yinxiaojian commented 5 years ago

Hello, I can't find the evaluation code in this project. For MAML, the evaluation need updae on the support set of test data, then evaluating on the query set of test data. Do you do the same process. Or do not need to update on the support set of test dataset, evaluate directly on the query set of test set.

Thank you!

hoyeoplee commented 5 years ago

We conducted the same process. As you mentioned, support sets are required during the test time. We used the forward method of MeLU class for the evaluation.