When i split the training set for dev, something strange happened！

hotpotqa / hotpot

Apache License 2.0

445 stars 75 forks source link

When i split the training set for dev, something strange happened！ #10

Closed luckysheep861 closed 5 years ago

luckysheep861 commented 5 years ago

I got an unbelievable high-performance when i split a part of training set for dev, certainly this part of data not be used in training. by the way, could you share the test set?

kimiyoung commented 5 years ago

All test set evaluation should be on our codalab server. This is to prevent overfitting the test set. See https://worksheets.codalab.org/worksheets/0xa8718c1a5e9e470e84a7d5fb3ab1dde2/ for instructions.

luckysheep861 commented 5 years ago

But i just find a solution solved comparison question. i cant complete all the tasks.

qipeng commented 5 years ago

Do you think you can use the dev set (which is public) for testing purposes? Not sure I'm understanding what the issue is here...