search

facebookresearch / Neural-Code-Search-Evaluation-Dataset

evaluation dataset consisting of natural language query and code snippet pairs
Other
123 stars 20 forks source link

can we calculate the MRR metric on the basis of NCS dataset? #3

Open Sewens opened 3 years ago

Sewens commented 3 years ago

I've noticed that the score_sheet only have the FRank score of each query which means the rank order of the first hit result. But the orginal paper have mentioned the MRR (in Table 1) of NCS and UNIF. So how can I calculate the MRR metric using NCS dataset.