Which one of "ndcg_cut_20" and "ndcg_cun_20 [interp]" is the result in reproduces document?

capreolus-ir / capreolus

A toolkit for end-to-end neural ad hoc retrieval

https://capreolus.ai

Apache License 2.0

95 stars 32 forks source link

Which one of "ndcg_cut_20" and "ndcg_cun_20 [interp]" is the result in reproduces document? #198

Closed yiyaxiaozhi closed 2 years ago

yiyaxiaozhi commented 2 years ago

I follow the instruction of https://github.com/capreolus-ir/capreolus/blob/master/docs/reproduction/PARADE.md to reproduce the PARADE result with extra layer on my way. After I train the modified PARADE cross 5 folds and obtain this message: Here are "ndcg_cut_20" and "ndcg_cun_20 [interp]", which one should be compared with the reported result of capreolus/docs/reproduction/PARADE.md? :

andrewyates commented 2 years ago

The reproduction document reports ndcg_cut_20 without interpolation ("ndcg_cut_20")