Open Zamony opened 3 years ago
Hi there! In the function corresponding to NDCG calculation I noticed that ignore_rank_of_non_annotated_urls flag is present. https://github.com/github/CodeSearchNet/blob/76a006fda591591f196fd0aef1d6282af3135f71/src/relevanceeval.py#L76
ignore_rank_of_non_annotated_urls
The question is: are your results in the paper calculated with ignore_rank_of_non_annotated_urls=True?
ignore_rank_of_non_annotated_urls=True
Both are reported. In Table 4, NDCG-Within for True, NDCG-All for False.
Hi there! In the function corresponding to NDCG calculation I noticed that
ignore_rank_of_non_annotated_urls
flag is present. https://github.com/github/CodeSearchNet/blob/76a006fda591591f196fd0aef1d6282af3135f71/src/relevanceeval.py#L76The question is: are your results in the paper calculated with
ignore_rank_of_non_annotated_urls=True
?