search

github / CodeSearchNet

Datasets, tools, and benchmarks for representation learning of code.
https://arxiv.org/abs/1909.09436
MIT License
2.17k stars 384 forks source link

NDCG computation #233

Open Zamony opened 3 years ago

Zamony commented 3 years ago

Hi there! In the function corresponding to NDCG calculation I noticed that ignore_rank_of_non_annotated_urls flag is present. https://github.com/github/CodeSearchNet/blob/76a006fda591591f196fd0aef1d6282af3135f71/src/relevanceeval.py#L76

The question is: are your results in the paper calculated with ignore_rank_of_non_annotated_urls=True?

Alex-HaochenLi commented 1 year ago

Both are reported. In Table 4, NDCG-Within for True, NDCG-All for False.