Lack of proper evaluation metrics

[jordane95]

I noticed that you reported accuracy@k and MRR in your paper. But I can't find corresponding implementation in your code.

Actually, I only find accuracy and f1, which are used to evaluate binary classification models.

[fewshotcdcs]

For the construction of search task, you can refer to section 3.5 in the paper. By selecting all data in the test set as candidates, the accuracy@k and MRR metrics can be calculated.

[jordane95]

Thanks for your reply!