Scripts/notebooks to reproduce the results reported in the paper?

princeton-nlp / LitSearch

[EMNLP 2024] A Retrieval Benchmark for Scientific Literature Search

MIT License

55 stars 3 forks source link

Scripts/notebooks to reproduce the results reported in the paper? #2

Open narayanacharya6 opened 1 month ago

narayanacharya6 commented 1 month ago

Are the scripts/notebooks to reproduce the results reported in the paper available? If yes, can they be added to this repository?

I see two functions in utils/utils.py to calculate recall/ndcg but that looks like computing for 1 query and not the aggregate score of the entire dataset.

gaotianyu1350 commented 2 weeks ago

Hi,

Thanks for your interest! I believe that the evaluation section of readme offered instructions on running the main evaluation numbers. Let us know if you actually mean something else and run into any problem!